-- Logs begin at Thu 2022-06-16 22:06:47 UTC, end at Thu 2022-06-16 23:18:22 UTC. -- Jun 16 22:46:31.323291 dv-cinder-846230-1-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 16 22:46:31.332618 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 16 22:46:31.334526 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 16 22:46:31 2022] *** Jun 16 22:46:31.334526 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 16 22:46:31.334526 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 16 22:46:31.334526 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: nodename: dv-cinder-846230-1-iscsi Jun 16 22:46:31.334526 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: machine: x86_64 Jun 16 22:46:31.334526 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: clock source: unix Jun 16 22:46:31.334526 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: pcre jit disabled Jun 16 22:46:31.334526 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: detected number of CPU cores: 4 Jun 16 22:46:31.334526 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: current working directory: / Jun 16 22:46:31.334526 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: detected binary path: /usr/bin/uwsgi-core Jun 16 22:46:31.334526 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: your processes number limit is 47680 Jun 16 22:46:31.334526 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: your memory page size is 4096 bytes Jun 16 22:46:31.334526 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: detected max file descriptor number: 2048 Jun 16 22:46:31.335194 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: lock engine: pthread robust mutexes Jun 16 22:46:31.335194 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: thunder lock: enabled Jun 16 22:46:31.335194 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 16 22:46:31.335194 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 16 22:46:31.378076 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: Python main interpreter initialized at 0x55cc95ab39c0 Jun 16 22:46:31.378076 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: python threads support enabled Jun 16 22:46:31.378076 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: your server socket listen backlog is limited to 100 connections Jun 16 22:46:31.378076 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: your mercy for graceful operations on workers is 90 seconds Jun 16 22:46:31.378428 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: mapped 403077 bytes (393 KB) for 2 cores Jun 16 22:46:31.378496 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: *** Operational MODE: preforking *** Jun 16 22:46:31.378780 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: *** uWSGI is running in multiple interpreter mode *** Jun 16 22:46:31.378780 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: spawned uWSGI master process (pid: 80075) Jun 16 22:46:31.378983 dv-cinder-846230-1-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 16 22:46:31.379757 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: spawned uWSGI worker 1 (pid: 80087, cores: 1) Jun 16 22:46:31.382636 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: spawned uWSGI worker 2 (pid: 80088, cores: 1) Jun 16 22:46:31.382636 dv-cinder-846230-1-iscsi devstack@keystone.service[80075]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 16 22:46:36.847135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 22:46:36.847135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: from cryptography.utils import int_from_bytes Jun 16 22:46:36.847931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 22:46:36.847931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: from cryptography.utils import int_from_bytes Jun 16 22:46:36.848341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 22:46:36.848341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: from cryptography.utils import int_from_bytes Jun 16 22:46:36.848448 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 22:46:36.848448 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: from cryptography.utils import int_from_bytes Jun 16 22:46:36.896869 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 22:46:36.897059 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 22:46:36.897149 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 22:46:36.897235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] command line args: [] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 22:46:36.897313 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 22:46:36.897509 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] command line args: [] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 22:46:36.897598 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 22:46:36.897671 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ================================================================================ {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 22:46:36.897784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 22:46:36.897922 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] admin_token = **** {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.898007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ================================================================================ {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 22:46:36.898167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] config_dir = [] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.898258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] admin_token = **** {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.898362 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.898448 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] config_dir = [] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.898545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] config_source = [] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.898652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.898752 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] debug = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.898841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] config_source = [] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.898950 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.899263 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] debug = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.899263 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.899487 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.899584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.899685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.899796 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.899895 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.900001 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.900103 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.900200 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.900311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] list_limit = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.900406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] list_limit = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.900507 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] log_config_append = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.900613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] log_config_append = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.900715 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.900823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.900919 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] log_dir = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.901019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] log_dir = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.901120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] log_file = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.901260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] log_file = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.901341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.901457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.901531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.901650 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.901763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.901846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.901918 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.901985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.902056 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.902171 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.902258 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.902331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.902455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.902530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.902680 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.902797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.902882 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.902959 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.903073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.903147 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.903266 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.903341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.903474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.903562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.990300 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.990482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.990663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.990740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.990923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.991005 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.991467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.991555 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.17/identity {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.991668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] publish_errors = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.991748 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] publish_errors = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.991917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.991995 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.992166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.992246 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.992355 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.992439 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.992616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.992715 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.992864 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.992948 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.993108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] standard_threads = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.993185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] standard_threads = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.993347 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.993493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.993601 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.993678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.993783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.993861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.993965 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] use_journal = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.994042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] use_journal = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.994145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] use_json = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.994228 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] use_json = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.994324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] use_stderr = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.994400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] use_stderr = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.994500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] use_syslog = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.994578 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] use_syslog = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.994762 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.994840 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 22:46:36.995032 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.995117 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.995278 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.995363 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.995546 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.995626 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.995791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.995872 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.996047 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.996134 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.996291 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.996374 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.996547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.996624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.996790 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.996868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.997042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.997119 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.997297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.997380 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.997553 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.997631 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.997793 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.997876 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.998043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.998120 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.998281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.998366 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.998533 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.998640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.998859 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.998949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.999101 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.999182 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.999352 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.999431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.999608 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.999683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.999858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:36.999935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.000095 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.000171 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.000353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.000437 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.000618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.000695 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.000875 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.000950 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.001130 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.001207 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.001364 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.001448 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.001598 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.001674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.001829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.001913 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.002068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.002144 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.002318 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.002393 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.002568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.002679 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.002828 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.002905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.003060 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.003136 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.003309 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.003392 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.003558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.003640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.003795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.003870 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.004044 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.004132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.004280 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.004362 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.004516 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.004591 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.004754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.004905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.004979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.005047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.005205 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.005282 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.005437 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.005521 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.005668 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.005750 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.005906 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.005990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.006153 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.006237 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.006386 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.006463 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.006643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.006733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.006880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.006964 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.007114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.007224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.007350 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.007453 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.007598 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.007710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.007815 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.007910 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.008056 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.008175 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.008311 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.008421 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.008526 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.008604 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.008742 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.008830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.008979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.009068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.009207 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.009284 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.009455 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.009536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.009689 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.009769 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.009929 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.010010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.010161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.010288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.010425 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.010544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.010666 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.010797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.010905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.010984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.011143 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.011227 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.011377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.011467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.011648 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.011727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.011888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.011968 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.012138 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.012219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.012386 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.012504 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.012626 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.012704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.012862 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.012941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.013099 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.013180 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.013345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.013422 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.013583 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.013670 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.013820 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.013899 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.014066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.014144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.014306 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.014385 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.014572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.014670 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.014837 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.014916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.015075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.015154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.015311 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.015389 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.015575 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.015695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.015821 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.015901 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.016062 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.016148 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.016300 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.016379 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.016528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.016604 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.016772 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.016858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.017007 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.017085 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.017243 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.017331 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.017472 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.017551 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.017702 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.017779 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.017931 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.018008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.018166 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.018245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.018679 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.018768 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.018925 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.019004 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.019163 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.019242 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.019411 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.019488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.019649 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.019726 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.019906 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.019985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.020170 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.020246 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.020412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.020497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.020642 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] auth.external = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.020797 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] auth.external = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.020878 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.020948 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.021156 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.021266 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.021392 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.021479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.021624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] auth.password = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.021702 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] auth.password = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.021856 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] auth.token = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.021934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] auth.token = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.022096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.022189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.022334 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.022412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.022580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.022715 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.022832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.022924 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.023081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.023162 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.023330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.023408 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.023563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.023655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.023799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] credential.caching = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.023878 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] credential.caching = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.024048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.024127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.024300 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.024387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.024551 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.024630 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.024788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.024867 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.025031 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.025121 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.025268 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.025346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.025519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.025603 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.025775 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.025861 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.026012 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.026092 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.026269 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.026348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.026550 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.026666 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.026824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.026912 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.027088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.027167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.027334 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.027422 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.027592 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.027672 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.027830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] federation.caching = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.027907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] federation.caching = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.028065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.028152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.028312 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.028435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.028558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.028637 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.028787 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.028876 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.029034 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.029121 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.029271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.029351 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.029525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.029630 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.029756 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.029841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.030534 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.030649 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.030808 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.030897 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.031059 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.031138 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.031296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] identity.caching = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.031375 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] identity.caching = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.031551 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.031730 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.031807 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.031879 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.032020 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.032099 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.032257 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.032338 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.032503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.032580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.032736 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.032813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.032971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.033055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.033221 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.033299 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.033474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.033552 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.033708 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.033788 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.033937 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.034021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.034167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.034245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.034409 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.034496 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.034684 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.034780 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.034939 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.035025 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.035194 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.035274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.035440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.035519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.035714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.035791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.035950 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.036030 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.036187 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.036267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.036418 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.036498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.036655 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.036735 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.036885 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.036963 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.037121 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.037198 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.037361 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.037441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.037602 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.037686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.037850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.037928 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.038082 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.038161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.038328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.038405 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.038576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.038677 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.038832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.038919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.039079 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.039165 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.039326 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.039405 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.039559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.039639 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.039796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.039874 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.040026 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.040103 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.040261 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.040340 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.040498 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.040577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.040743 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.040821 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.040983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.041063 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.041218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.041295 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.041468 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.041545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.041716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.041796 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.041950 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.042027 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.042179 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.042256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.042428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.042508 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.042698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.042782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.042937 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.043017 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.043172 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.043252 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.043407 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.043485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.043640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.043726 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.043879 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.043957 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.044140 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.044219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.044373 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.044460 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.044621 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.044700 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.044870 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.044946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.045115 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.045201 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.045351 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.045430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.045582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.045659 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.046374 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.046468 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.046637 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.046719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.046884 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.046962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.047116 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.047204 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.047367 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.047447 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.047617 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.047695 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.047862 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.047949 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.048110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.048190 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.048359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.048439 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.048591 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.048678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.048839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.048918 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.049077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.049154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.049311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.049402 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.049548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.049627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.049784 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.049863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.050029 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.050117 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.050277 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.050356 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.050514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.050636 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.050793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.050885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.051025 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.051107 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.051271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.051350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.051506 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.051592 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.051742 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.051820 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.051977 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.052054 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.052224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.052309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.052463 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.052542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.052695 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.052774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.052931 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.053019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.053172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] resource.caching = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.053249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] resource.caching = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.053420 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.053498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.053669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.053756 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.053903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.053980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.054153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.054231 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.054401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.054488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.054656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.054738 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.054908 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.054985 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.055144 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.055304 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.055381 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.055452 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.055607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] role.caching = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.055687 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] role.caching = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.055857 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] role.driver = sql {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.055943 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] role.driver = sql {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.056086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.056170 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.056334 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.056413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.056582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.056670 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.056828 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.056906 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.057075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.057153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.057324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.057404 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.057568 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.057646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.057817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.057896 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.058067 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.058146 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.058297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.058375 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.058545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.058663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.058818 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.058900 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.059068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.059145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.059312 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.059389 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.059711 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.059792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.059951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.060029 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.060202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.060281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.060453 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.060534 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.060704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.060782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.060955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.061040 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.061188 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.061268 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.061472 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.061550 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.061735 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.061814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.061981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.062062 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.062212 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.062290 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.062446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.062532 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.062683 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.062757 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.062941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.063018 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.063198 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.063299 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.063448 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.063528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.063686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.063763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.063920 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.064006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.064153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.064230 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.064387 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] token.caching = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.064464 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] token.caching = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.064623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.064713 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.064872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.064951 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.065113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.065190 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.065371 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.065456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.065620 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.065699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.065858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.065935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.066101 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.066187 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.066346 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.066423 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.066622 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.066703 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.066854 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.066944 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.067095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.067173 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.067328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.067405 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.067575 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.067661 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.067826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.067905 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.068057 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.068134 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.068299 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.068386 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.068594 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.068672 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.068858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.connection = **** {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.068945 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.connection = **** {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.069106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.069194 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.069352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.069431 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.069591 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.069672 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.069832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.069910 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.070069 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.070147 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.070304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.070384 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.070541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.070636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.070803 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.070881 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.071049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.071130 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.071285 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.071365 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.071530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.071608 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.071763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.071843 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.072010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.072089 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.072245 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.072323 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.072487 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.072567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.072717 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.072794 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.072953 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.073031 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.073189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.073268 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 22:46:37.073340 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80087) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 22:46:37.073410 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80088) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 22:46:37.103105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.103233 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.103326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.103403 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.103476 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.103550 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.111886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.112379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.140691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.140906 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.140999 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.141135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.194208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.194364 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.243141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.243376 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.243959 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.244200 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.308691 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.308918 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.309031 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.309135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.309266 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.309348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.309491 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.309573 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.309714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.309804 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.309935 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.310153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.333090 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.333306 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.333406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.333531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.333613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.333754 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.333834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.333973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.334055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.334189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.334269 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.334403 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.334482 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.334639 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.345378 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.345688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.357866 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.358101 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.358886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.358976 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.359108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.359188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.359325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.359418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.359545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.359624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.359756 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80087) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.359836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.359965 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.360179 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80088) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 22:46:37.377283 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.377492 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 22:46:37.377692 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.377823 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.377918 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 22:46:37.378022 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 22:46:37.378224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.378303 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.378429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 22:46:37.378535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.378729 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.378810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.378992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.379070 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.379251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.379508 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.389916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.390191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.390375 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.390456 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.390687 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.390783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.390959 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.391239 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.399228 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80087) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.399769 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.402508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80087) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.402809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.403088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.403212 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.403364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.403498 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.403640 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.403767 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.403896 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.404022 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.404157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.404284 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.404409 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.404535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.404660 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.404786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.404930 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.405057 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.405192 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.405318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.405469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.405737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.432210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.432826 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.433828 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80087) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.434103 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.434447 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.434754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.439994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80087) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.440384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.440577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.440709 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.440970 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.441253 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.447956 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80087) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.448235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.448601 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.448885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.452790 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80087) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.453071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.453349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.453476 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.453615 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.453744 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.453870 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.453995 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.454122 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.454249 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.454382 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.454519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.454666 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.454792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.455013 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.455279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.473331 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.473603 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.474039 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.474317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.478242 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80087) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.478931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.481903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.482295 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.482620 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.483008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.487855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80087) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.488549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.491513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80087) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.492349 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.495156 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.495455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.495737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.496018 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.496145 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.496276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.496416 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.496547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.496678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.496848 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.497125 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.497403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.518255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80087) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.518801 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.519079 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.519342 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.522401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.522712 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.522990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.523256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.527209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.528917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.529197 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.532947 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80087) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.533235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.533515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.533805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.534131 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.534543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.537420 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.537871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.541465 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.541759 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.542042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.542340 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.542636 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.548348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80087) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.548624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.548918 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.549197 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.549567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.549851 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.550128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.556673 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.556957 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.557248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.557529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.557903 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.558183 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.558462 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.568024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80087) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.571277 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80087) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.574575 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.576308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.576640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.578003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80087) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.578279 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.578553 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.579954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.583281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.585028 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.585813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.586704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.586987 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.587269 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.587461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80087) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.590707 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.590995 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.591381 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.591660 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.595199 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.596819 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.600092 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.600383 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.600770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.601048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.601334 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80087) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.601619 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.601899 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.610381 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80087) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.610628 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.610750 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.610908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.611037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.611178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.611315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.611446 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.611840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.612114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.612392 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.612667 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.612932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.613197 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.613456 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.613726 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.613988 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80087) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.619219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80088) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 22:46:37.619500 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.619782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.620060 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.620334 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.620726 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.621003 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.621282 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.621562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.621833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.622104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.622368 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.622659 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.622933 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80088) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 22:46:37.651712 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cc95ab39c0 pid: 80087 (default app) Jun 16 22:46:37.655074 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2fd6c-6ff3-4a57-a9ad-4e6dc65574ee None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:37.655923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2fd6c-6ff3-4a57-a9ad-4e6dc65574ee None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:37.656204 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2fd6c-6ff3-4a57-a9ad-4e6dc65574ee None None] PATH_INFO: `/v3` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:37.660408 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1/1] 10.222.0.17 () {50 vars in 791 bytes} [Thu Jun 16 22:46:37 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:46:37.661296 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55cc95ab39c0 pid: 80088 (default app) Jun 16 22:46:45.360747 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90bbc9-47fa-4990-b447-79aa8c516456 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:45.361284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90bbc9-47fa-4990-b447-79aa8c516456 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:45.361284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90bbc9-47fa-4990-b447-79aa8c516456 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:45.362693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 2/2] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:46:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:46:45.370056 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe0d29-db2a-49bd-b462-88f39cf32c44 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:45.370476 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe0d29-db2a-49bd-b462-88f39cf32c44 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:45.370640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe0d29-db2a-49bd-b462-88f39cf32c44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:45.440216 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: INFO dbcounter [None req-96fe0d29-db2a-49bd-b462-88f39cf32c44 None None] Registered counter for database keystone Jun 16 22:46:45.460863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG oslo_db.sqlalchemy.engines [None req-96fe0d29-db2a-49bd-b462-88f39cf32c44 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80088) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 22:46:45.466925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writer thread running {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 22:46:45.643042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG passlib.handlers.bcrypt [None req-96fe0d29-db2a-49bd-b462-88f39cf32c44 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80088) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 22:46:45.643461 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG passlib.handlers.bcrypt [None req-96fe0d29-db2a-49bd-b462-88f39cf32c44 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80088) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 22:46:45.697711 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-96fe0d29-db2a-49bd-b462-88f39cf32c44 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:46:45.699344 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-96fe0d29-db2a-49bd-b462-88f39cf32c44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:46:45.821872 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1/3] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:46:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jun 16 22:46:45.854197 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90bbc9-47fa-4990-b447-79aa8c516456 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:45.924763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: INFO dbcounter [None req-8f90bbc9-47fa-4990-b447-79aa8c516456 None None] Registered counter for database keystone Jun 16 22:46:45.944263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG oslo_db.sqlalchemy.engines [None req-8f90bbc9-47fa-4990-b447-79aa8c516456 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80087) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 22:46:45.951199 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writer thread running {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 22:46:46.069534 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90bbc9-47fa-4990-b447-79aa8c516456 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:46.073781 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbcac54-57fe-4385-a198-090255264ffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:46.074442 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcac54-57fe-4385-a198-090255264ffd admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:46.074764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcac54-57fe-4385-a198-090255264ffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:46.075049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbcac54-57fe-4385-a198-090255264ffd admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:46.083136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbcac54-57fe-4385-a198-090255264ffd admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:46.134886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: WARNING keystone.common.rbac_enforcer.enforcer [None req-efbcac54-57fe-4385-a198-090255264ffd admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 22:46:46.135245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbcac54-57fe-4385-a198-090255264ffd admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:46.144776 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 3/4] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jun 16 22:46:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 22:46:46.151045 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe0d29-db2a-49bd-b462-88f39cf32c44 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:46.165259 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe0d29-db2a-49bd-b462-88f39cf32c44 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:46.169589 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669466e1-56df-4cb7-aa6c-e44c422eb83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:46.170745 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669466e1-56df-4cb7-aa6c-e44c422eb83a admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:46.171036 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669466e1-56df-4cb7-aa6c-e44c422eb83a admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:46.171315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669466e1-56df-4cb7-aa6c-e44c422eb83a admin admin] PATH_INFO: `/v3/projects/2ffbb7bcd828426c857e7fce279519f7` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:46.173305 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669466e1-56df-4cb7-aa6c-e44c422eb83a admin admin] RBAC: Authorizing `identity:get_project(project_id=2ffbb7bcd828426c857e7fce279519f7)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:46.223258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: WARNING keystone.common.rbac_enforcer.enforcer [None req-669466e1-56df-4cb7-aa6c-e44c422eb83a admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 22:46:46.223543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669466e1-56df-4cb7-aa6c-e44c422eb83a admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:46.228142 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 2/5] 10.222.0.17 () {56 vars in 1207 bytes} [Thu Jun 16 22:46:46 2022] GET /identity/v3/projects/2ffbb7bcd828426c857e7fce279519f7 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:46:48.292798 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed13b7-a0cf-4c6e-8416-def40f80b49d None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:48.293310 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed13b7-a0cf-4c6e-8416-def40f80b49d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:48.293393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ed13b7-a0cf-4c6e-8416-def40f80b49d None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:48.297450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 4/6] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:46:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:46:48.302715 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e846a-6f70-4608-bcf6-ea6083cbf405 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:48.303067 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e846a-6f70-4608-bcf6-ea6083cbf405 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:48.303289 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7e846a-6f70-4608-bcf6-ea6083cbf405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:48.322249 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-1e7e846a-6f70-4608-bcf6-ea6083cbf405 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:46:48.323826 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-1e7e846a-6f70-4608-bcf6-ea6083cbf405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:46:48.332311 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 3/7] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:46:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 22:46:48.353758 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed13b7-a0cf-4c6e-8416-def40f80b49d None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:48.365850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ed13b7-a0cf-4c6e-8416-def40f80b49d None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:48.370059 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfcb180-a84f-4e76-952c-3c308e1fcf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:48.370735 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcb180-a84f-4e76-952c-3c308e1fcf56 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:48.371032 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcb180-a84f-4e76-952c-3c308e1fcf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:48.371314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfcb180-a84f-4e76-952c-3c308e1fcf56 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:48.379606 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcb180-a84f-4e76-952c-3c308e1fcf56 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:48.381383 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfcb180-a84f-4e76-952c-3c308e1fcf56 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:48.390394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 5/8] 10.222.0.17 () {58 vars in 1324 bytes} [Thu Jun 16 22:46:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 22:46:48.396903 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e846a-6f70-4608-bcf6-ea6083cbf405 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:48.406547 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7e846a-6f70-4608-bcf6-ea6083cbf405 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:48.411030 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d81d5a1-d15a-4954-8024-9e923f40054c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:48.411899 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81d5a1-d15a-4954-8024-9e923f40054c admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:48.412190 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81d5a1-d15a-4954-8024-9e923f40054c admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:48.412477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d81d5a1-d15a-4954-8024-9e923f40054c admin admin] PATH_INFO: `/v3/users/1288d3a7956c42f7ad18e2e126c7dd07` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:48.413883 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81d5a1-d15a-4954-8024-9e923f40054c admin admin] RBAC: Authorizing `identity:get_user(user_id=1288d3a7956c42f7ad18e2e126c7dd07)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:48.415754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d81d5a1-d15a-4954-8024-9e923f40054c admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:48.417335 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 4/9] 10.222.0.17 () {56 vars in 1198 bytes} [Thu Jun 16 22:46:48 2022] GET /identity/v3/users/1288d3a7956c42f7ad18e2e126c7dd07 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:46:50.572971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f0b94-c183-479a-ae61-7ed5ff286f35 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:50.573468 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f0b94-c183-479a-ae61-7ed5ff286f35 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:50.573468 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546f0b94-c183-479a-ae61-7ed5ff286f35 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:50.574838 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 6/10] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:46:50.579951 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98333a67-e882-4918-861b-4cab4e10b059 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:50.580354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98333a67-e882-4918-861b-4cab4e10b059 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:50.580517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98333a67-e882-4918-861b-4cab4e10b059 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:50.591349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7817e0-6fca-492a-9ab1-27f71bcb23fa None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:50.591642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7817e0-6fca-492a-9ab1-27f71bcb23fa None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:50.591925 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7817e0-6fca-492a-9ab1-27f71bcb23fa None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:50.593253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 7/11] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:46:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:46:50.598731 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2006ffd-9e30-4671-b99e-771d796b17cd None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:50.598857 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-98333a67-e882-4918-861b-4cab4e10b059 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:46:50.599030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2006ffd-9e30-4671-b99e-771d796b17cd None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:50.599318 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2006ffd-9e30-4671-b99e-771d796b17cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:50.599938 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-98333a67-e882-4918-861b-4cab4e10b059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:46:50.640412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG passlib.handlers.bcrypt [None req-b2006ffd-9e30-4671-b99e-771d796b17cd None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80087) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 22:46:50.640824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG passlib.handlers.bcrypt [None req-b2006ffd-9e30-4671-b99e-771d796b17cd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80087) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 22:46:50.679782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 5/12] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:46:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:46:50.687794 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-b2006ffd-9e30-4671-b99e-771d796b17cd None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:46:50.689433 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b2006ffd-9e30-4671-b99e-771d796b17cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:46:50.706048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98333a67-e882-4918-861b-4cab4e10b059 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:50.729484 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:50.788130 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 8/13] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:46:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:46:50.807881 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98333a67-e882-4918-861b-4cab4e10b059 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:50.809700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2006ffd-9e30-4671-b99e-771d796b17cd None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:50.842303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b2ba8-2830-4b4a-af08-4cbd0c05c326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:50.843082 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b2ba8-2830-4b4a-af08-4cbd0c05c326 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:50.843378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b2ba8-2830-4b4a-af08-4cbd0c05c326 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:50.843652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b2ba8-2830-4b4a-af08-4cbd0c05c326 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:50.855053 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b2ba8-2830-4b4a-af08-4cbd0c05c326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:46:50.866651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-3e2b2ba8-2830-4b4a-af08-4cbd0c05c326 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:46:50.866651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:46:50.866651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:46:50.866651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:46:50.866651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:46:50.866651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:46:50.866651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:46:50.866651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:46:50.866651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:46:50.866651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:46:50.866651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:46:50.866651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:46:50.866651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:46:50.866651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:46:50.866651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:46:50.866651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:46:50.867510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:46:50.867510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:46:50.867510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:46:50.867510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:46:50.867510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:46:50.867510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:46:50.867510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:46:50.867510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:46:50.867510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:46:50.867510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:46:50.867510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:46:50.867510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:46:50.867510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:46:50.867510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:46:50.867510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:46:50.867510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:46:50.868342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:46:50.868342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:46:50.868342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:46:50.868342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:46:50.868342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:46:50.868342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:46:50.868342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:46:50.868342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:46:50.868342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:46:50.868342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:46:50.868342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:46:50.868342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:46:50.868342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 6/14] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:46:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:46:50.873448 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b2ba8-2830-4b4a-af08-4cbd0c05c326 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:50.909881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2006ffd-9e30-4671-b99e-771d796b17cd None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:50.912149 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:50.948718 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f99deb-1a96-496d-8599-a9d2a0641f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:50.949622 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f99deb-1a96-496d-8599-a9d2a0641f2e None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:50.949913 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f99deb-1a96-496d-8599-a9d2a0641f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:50.950194 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f99deb-1a96-496d-8599-a9d2a0641f2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:50.955441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:50.973276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b2ba8-2830-4b4a-af08-4cbd0c05c326 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:51.007425 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bc12d-ed4a-4bce-8d62-272bc39c0381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:51.008132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bc12d-ed4a-4bce-8d62-272bc39c0381 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:51.008428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bc12d-ed4a-4bce-8d62-272bc39c0381 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:51.008710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344bc12d-ed4a-4bce-8d62-272bc39c0381 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:51.009492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bc12d-ed4a-4bce-8d62-272bc39c0381 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:46:51.009806 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bc12d-ed4a-4bce-8d62-272bc39c0381 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:51.011246 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344bc12d-ed4a-4bce-8d62-272bc39c0381 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:51.018406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f99deb-1a96-496d-8599-a9d2a0641f2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:51.019946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f99deb-1a96-496d-8599-a9d2a0641f2e None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:51.023033 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 7/15] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:46:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:46:51.028546 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bc12d-ed4a-4bce-8d62-272bc39c0381 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:51.116546 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 9/16] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:46:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 22:46:51.122134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f99deb-1a96-496d-8599-a9d2a0641f2e None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:51.124750 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:51.128137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344bc12d-ed4a-4bce-8d62-272bc39c0381 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:51.163376 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56d04d-f324-4733-8d78-7550a1765376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:51.164262 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56d04d-f324-4733-8d78-7550a1765376 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:51.164556 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56d04d-f324-4733-8d78-7550a1765376 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:51.164830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56d04d-f324-4733-8d78-7550a1765376 None admin] PATH_INFO: `/v3/users/1288d3a7956c42f7ad18e2e126c7dd07` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:51.166030 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56d04d-f324-4733-8d78-7550a1765376 None admin] RBAC: Authorizing `identity:get_user(user_id=1288d3a7956c42f7ad18e2e126c7dd07)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:51.167516 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56d04d-f324-4733-8d78-7550a1765376 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:51.169132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 8/17] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:46:51 2022] GET /identity/v3/users/1288d3a7956c42f7ad18e2e126c7dd07 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:46:51.174964 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56d04d-f324-4733-8d78-7550a1765376 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:51.188855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:51.221528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f99deb-1a96-496d-8599-a9d2a0641f2e None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:51.256414 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea668f99-a1cb-4415-9dca-5ba7397cb63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:51.257220 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea668f99-a1cb-4415-9dca-5ba7397cb63d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:51.257503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea668f99-a1cb-4415-9dca-5ba7397cb63d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:51.257781 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea668f99-a1cb-4415-9dca-5ba7397cb63d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:51.272738 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56d04d-f324-4733-8d78-7550a1765376 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:51.276067 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea668f99-a1cb-4415-9dca-5ba7397cb63d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:51.277542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea668f99-a1cb-4415-9dca-5ba7397cb63d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:51.298327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-ea668f99-a1cb-4415-9dca-5ba7397cb63d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:46:51.298327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:46:51.298327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:46:51.298327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:46:51.298327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:46:51.298327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:46:51.298327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:46:51.298327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:46:51.298327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:46:51.298327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:46:51.298327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:46:51.298327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:46:51.298327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:46:51.298327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:46:51.298327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:46:51.299810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:46:51.299810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:46:51.299810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:46:51.299810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:46:51.299810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:46:51.299810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:46:51.299810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:46:51.299810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:46:51.299810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:46:51.299810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:46:51.299810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:46:51.299810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:46:51.299810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:46:51.299810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:46:51.299810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:46:51.299810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:46:51.299810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:46:51.300589 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:46:51.300589 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:46:51.300589 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:46:51.300589 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:46:51.300589 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 10/18] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:46:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:46:51.305864 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea668f99-a1cb-4415-9dca-5ba7397cb63d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:51.306950 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964f8c6-6a0a-4d31-a1f4-c5fab7676c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:51.307778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964f8c6-6a0a-4d31-a1f4-c5fab7676c7a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:51.308068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964f8c6-6a0a-4d31-a1f4-c5fab7676c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:51.308380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2964f8c6-6a0a-4d31-a1f4-c5fab7676c7a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:51.309561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964f8c6-6a0a-4d31-a1f4-c5fab7676c7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:51.311040 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2964f8c6-6a0a-4d31-a1f4-c5fab7676c7a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:51.312637 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 9/19] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:46:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:46:51.318148 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964f8c6-6a0a-4d31-a1f4-c5fab7676c7a None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:51.360489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:51.398162 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:51.405077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea668f99-a1cb-4415-9dca-5ba7397cb63d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:51.416658 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2964f8c6-6a0a-4d31-a1f4-c5fab7676c7a None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:51.440368 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd9b850-2024-4ec9-8015-d32e4990c9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:51.441232 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9b850-2024-4ec9-8015-d32e4990c9ed None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:51.441523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9b850-2024-4ec9-8015-d32e4990c9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:51.441804 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd9b850-2024-4ec9-8015-d32e4990c9ed None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:51.442528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd9b850-2024-4ec9-8015-d32e4990c9ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:46:51.442872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd9b850-2024-4ec9-8015-d32e4990c9ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:51.444256 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cd9b850-2024-4ec9-8015-d32e4990c9ed None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:51.450545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1932d7f-88a5-4446-b49b-1ae74e6484c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:51.451226 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1932d7f-88a5-4446-b49b-1ae74e6484c1 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:51.451512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1932d7f-88a5-4446-b49b-1ae74e6484c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:51.451793 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1932d7f-88a5-4446-b49b-1ae74e6484c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:51.452643 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1932d7f-88a5-4446-b49b-1ae74e6484c1 None admin] RBAC: Adding query filter params (role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.domain.id=default, user.id=1288d3a7956c42f7ad18e2e126c7dd07) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:46:51.452955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1932d7f-88a5-4446-b49b-1ae74e6484c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:51.454373 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1932d7f-88a5-4446-b49b-1ae74e6484c1 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:51.460940 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 10/20] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jun 16 22:46:51 2022] GET /identity/v3/role_assignments?user.id=1288d3a7956c42f7ad18e2e126c7dd07&scope.domain.id=default&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 233 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:46:51.461868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 11/21] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:46:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:46:53.961843 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e306cea-8973-4a18-a6ea-4e8b93b482cf None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:53.962373 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e306cea-8973-4a18-a6ea-4e8b93b482cf None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:53.962449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e306cea-8973-4a18-a6ea-4e8b93b482cf None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:53.963807 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 11/22] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:46:53.969338 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13939a0-ce0d-4033-864e-01ec646fafd0 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:53.969720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13939a0-ce0d-4033-864e-01ec646fafd0 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:53.969907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13939a0-ce0d-4033-864e-01ec646fafd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:53.991626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-e13939a0-ce0d-4033-864e-01ec646fafd0 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:46:53.992855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e13939a0-ce0d-4033-864e-01ec646fafd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:46:54.088002 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 12/23] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:46:54.090644 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21278b-fbe9-4971-9341-52e843e7e72c None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:54.090949 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21278b-fbe9-4971-9341-52e843e7e72c None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:54.091234 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b21278b-fbe9-4971-9341-52e843e7e72c None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:54.092585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 12/24] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:46:54.098263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25ab49-c5d2-48af-9dec-084e13ed7b6c None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:54.098568 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25ab49-c5d2-48af-9dec-084e13ed7b6c None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:54.098886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f25ab49-c5d2-48af-9dec-084e13ed7b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:54.119960 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189c335-1341-4013-9782-670197fadbf0 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:54.120279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189c335-1341-4013-9782-670197fadbf0 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:54.120532 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e189c335-1341-4013-9782-670197fadbf0 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:54.121940 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 13/25] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:46:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:46:54.123213 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-8f25ab49-c5d2-48af-9dec-084e13ed7b6c None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:46:54.124626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-8f25ab49-c5d2-48af-9dec-084e13ed7b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:46:54.135189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189c335-1341-4013-9782-670197fadbf0 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:54.175861 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:54.202204 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 13/26] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:46:54.211343 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a2afb-dc43-43c6-90ab-1ec81a70a966 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:54.211943 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a2afb-dc43-43c6-90ab-1ec81a70a966 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:54.212560 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6a2afb-dc43-43c6-90ab-1ec81a70a966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:54.239068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-da6a2afb-dc43-43c6-90ab-1ec81a70a966 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:46:54.246641 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-da6a2afb-dc43-43c6-90ab-1ec81a70a966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:46:54.370743 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 14/27] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:46:54.371641 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a2afb-dc43-43c6-90ab-1ec81a70a966 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:54.372892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e189c335-1341-4013-9782-670197fadbf0 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:54.410498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e6698d-45aa-4996-b4bf-4d74a4de0157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:54.411711 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6698d-45aa-4996-b4bf-4d74a4de0157 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:54.412218 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6698d-45aa-4996-b4bf-4d74a4de0157 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:54.412697 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e6698d-45aa-4996-b4bf-4d74a4de0157 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:54.426224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6698d-45aa-4996-b4bf-4d74a4de0157 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:54.428113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e6698d-45aa-4996-b4bf-4d74a4de0157 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:54.431318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:54.450009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-d6e6698d-45aa-4996-b4bf-4d74a4de0157 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:46:54.450009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:46:54.450009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:46:54.450009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:46:54.450009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:46:54.450009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:46:54.450009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:46:54.450009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:46:54.450009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:46:54.450009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:46:54.450009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:46:54.450009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:46:54.450009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:46:54.450009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:46:54.450009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:46:54.450887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:46:54.450887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:46:54.450887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:46:54.450887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:46:54.450887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:46:54.450887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:46:54.450887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:46:54.450887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:46:54.450887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:46:54.450887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:46:54.450887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:46:54.450887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:46:54.450887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:46:54.450887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:46:54.450887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:46:54.450887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:46:54.450887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:46:54.451646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:46:54.451646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:46:54.451646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:46:54.451646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:46:54.453578 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 14/28] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:46:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:46:54.455123 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c7790-14cf-453b-af8a-25d07b23d4c8 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:54.455611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c7790-14cf-453b-af8a-25d07b23d4c8 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:54.456606 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645c7790-14cf-453b-af8a-25d07b23d4c8 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:54.458103 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 15/29] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:46:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:46:54.458992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c7790-14cf-453b-af8a-25d07b23d4c8 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:54.514562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6a2afb-dc43-43c6-90ab-1ec81a70a966 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:54.525844 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:54.563080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9576f-cc96-4fbe-a8dd-b82247e400cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:54.563887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9576f-cc96-4fbe-a8dd-b82247e400cb None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:54.564174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9576f-cc96-4fbe-a8dd-b82247e400cb None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:54.564695 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c9576f-cc96-4fbe-a8dd-b82247e400cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:54.569607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645c7790-14cf-453b-af8a-25d07b23d4c8 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:54.578849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c9576f-cc96-4fbe-a8dd-b82247e400cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:46:54.591036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-d2c9576f-cc96-4fbe-a8dd-b82247e400cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:46:54.591036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:46:54.591036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:46:54.591036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:46:54.591036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:46:54.591036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:46:54.591036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:46:54.591036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:46:54.591036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:46:54.591036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:46:54.591036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:46:54.591036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:46:54.591036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:46:54.591036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:46:54.591036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:46:54.591036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:46:54.591909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:46:54.591909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:46:54.591909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:46:54.591909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:46:54.591909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:46:54.591909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:46:54.591909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:46:54.591909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:46:54.591909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:46:54.591909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:46:54.591909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:46:54.591909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:46:54.591909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:46:54.591909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:46:54.591909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:46:54.591909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:46:54.592653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:46:54.592653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:46:54.592653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:46:54.592653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:46:54.592653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:46:54.592653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:46:54.592653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:46:54.592653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:46:54.592653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:46:54.592653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:46:54.592653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:46:54.592653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:46:54.592653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 15/30] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:46:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:46:54.593257 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9576f-cc96-4fbe-a8dd-b82247e400cb None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:54.617430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e521a-cd18-4115-b406-a23cca7aadb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:54.618568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e521a-cd18-4115-b406-a23cca7aadb6 None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:54.618903 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e521a-cd18-4115-b406-a23cca7aadb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:54.619184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4e521a-cd18-4115-b406-a23cca7aadb6 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:54.620194 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e521a-cd18-4115-b406-a23cca7aadb6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:54.621905 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4e521a-cd18-4115-b406-a23cca7aadb6 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:54.715243 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c9576f-cc96-4fbe-a8dd-b82247e400cb None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:54.751579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57010ec6-3f05-4046-9471-2b5e53dc3883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:54.752294 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57010ec6-3f05-4046-9471-2b5e53dc3883 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:54.752730 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57010ec6-3f05-4046-9471-2b5e53dc3883 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:54.753539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57010ec6-3f05-4046-9471-2b5e53dc3883 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:54.754460 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57010ec6-3f05-4046-9471-2b5e53dc3883 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:46:54.754814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57010ec6-3f05-4046-9471-2b5e53dc3883 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:54.756236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57010ec6-3f05-4046-9471-2b5e53dc3883 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:54.769691 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 16/31] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:46:54.771352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d56007-c42c-4acc-b5c9-2df02992427d None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:54.771649 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d56007-c42c-4acc-b5c9-2df02992427d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:54.771930 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d56007-c42c-4acc-b5c9-2df02992427d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:54.791859 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-85d56007-c42c-4acc-b5c9-2df02992427d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:46:54.793140 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-85d56007-c42c-4acc-b5c9-2df02992427d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:46:54.818399 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:54.869639 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 16/32] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jun 16 22:46:54 2022] POST /identity/v3/roles => generated 219 bytes in 411 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:46:54.871110 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e521a-cd18-4115-b406-a23cca7aadb6 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:54.876987 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 17/33] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:46:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:46:54.878714 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d56007-c42c-4acc-b5c9-2df02992427d None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:55.001332 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:55.051007 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d56007-c42c-4acc-b5c9-2df02992427d None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:55.104308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1aa30b8-7054-4ac7-b010-a6eb71bd25e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:55.105359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa30b8-7054-4ac7-b010-a6eb71bd25e3 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:55.105843 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa30b8-7054-4ac7-b010-a6eb71bd25e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:55.106317 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1aa30b8-7054-4ac7-b010-a6eb71bd25e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:55.107754 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa30b8-7054-4ac7-b010-a6eb71bd25e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:55.109949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1aa30b8-7054-4ac7-b010-a6eb71bd25e3 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:55.120722 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4e521a-cd18-4115-b406-a23cca7aadb6 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:55.167040 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858300b-013b-49ec-ac21-ba6052ceaa21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:55.167957 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858300b-013b-49ec-ac21-ba6052ceaa21 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:55.168246 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858300b-013b-49ec-ac21-ba6052ceaa21 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:55.168528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4858300b-013b-49ec-ac21-ba6052ceaa21 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:55.169289 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858300b-013b-49ec-ac21-ba6052ceaa21 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:46:55.169599 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858300b-013b-49ec-ac21-ba6052ceaa21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:55.172065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4858300b-013b-49ec-ac21-ba6052ceaa21 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:55.181928 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 17/34] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:46:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:46:55.182873 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858300b-013b-49ec-ac21-ba6052ceaa21 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:55.309316 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4858300b-013b-49ec-ac21-ba6052ceaa21 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:55.333374 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:55.353559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d131c7-eb6c-45f0-9f27-7d68f83d305f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:55.354664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d131c7-eb6c-45f0-9f27-7d68f83d305f None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:55.355317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d131c7-eb6c-45f0-9f27-7d68f83d305f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:55.355734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d131c7-eb6c-45f0-9f27-7d68f83d305f None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:55.356638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d131c7-eb6c-45f0-9f27-7d68f83d305f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:55.358216 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d131c7-eb6c-45f0-9f27-7d68f83d305f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:55.378863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 18/35] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jun 16 22:46:55 2022] POST /identity/v3/roles => generated 225 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:46:55.380212 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d131c7-eb6c-45f0-9f27-7d68f83d305f None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:55.430195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 18/36] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 22:46:54 2022] POST /identity/v3/projects => generated 301 bytes in 552 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:46:55.513998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d131c7-eb6c-45f0-9f27-7d68f83d305f None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:55.546781 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db118dd9-7373-4a56-b67d-d8da5feaa7ee None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:55.547442 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db118dd9-7373-4a56-b67d-d8da5feaa7ee None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:55.547923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db118dd9-7373-4a56-b67d-d8da5feaa7ee None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:55.549956 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 19/37] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:46:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:46:55.555821 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e336737-e78a-47ac-9a48-e74f6afd066c None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:55.556465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e336737-e78a-47ac-9a48-e74f6afd066c None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:55.557065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e336737-e78a-47ac-9a48-e74f6afd066c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:55.557475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f727e1e-9e62-4331-97b3-11e5ccf20854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:55.558478 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f727e1e-9e62-4331-97b3-11e5ccf20854 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:55.561891 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f727e1e-9e62-4331-97b3-11e5ccf20854 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:55.561891 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f727e1e-9e62-4331-97b3-11e5ccf20854 None admin] PATH_INFO: `/v3/users/1288d3a7956c42f7ad18e2e126c7dd07` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:55.562660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f727e1e-9e62-4331-97b3-11e5ccf20854 None admin] RBAC: Authorizing `identity:get_user(user_id=1288d3a7956c42f7ad18e2e126c7dd07)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:55.564209 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f727e1e-9e62-4331-97b3-11e5ccf20854 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:55.566746 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 19/38] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:46:55 2022] GET /identity/v3/users/1288d3a7956c42f7ad18e2e126c7dd07 => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:46:55.568382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81736d-cbbb-45f2-b044-96895a877220 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:55.568857 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81736d-cbbb-45f2-b044-96895a877220 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:55.569143 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81736d-cbbb-45f2-b044-96895a877220 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:55.570512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 20/39] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:46:55.575211 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81736d-cbbb-45f2-b044-96895a877220 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:55.585269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-0e336737-e78a-47ac-9a48-e74f6afd066c None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:46:55.587303 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-0e336737-e78a-47ac-9a48-e74f6afd066c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:46:55.607231 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:55.647321 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:55.682716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 20/40] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:46:55.684314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4058ba-056b-49e9-99c4-71aa420b8f1b None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:55.684618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4058ba-056b-49e9-99c4-71aa420b8f1b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:55.684902 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4058ba-056b-49e9-99c4-71aa420b8f1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:55.705123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-ee4058ba-056b-49e9-99c4-71aa420b8f1b None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:46:55.707294 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ee4058ba-056b-49e9-99c4-71aa420b8f1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:46:55.727863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81736d-cbbb-45f2-b044-96895a877220 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:55.763808 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911af5a-5141-4a46-a1fd-86e34cfe6207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:55.764612 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911af5a-5141-4a46-a1fd-86e34cfe6207 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:55.764906 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911af5a-5141-4a46-a1fd-86e34cfe6207 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:55.765192 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911af5a-5141-4a46-a1fd-86e34cfe6207 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:55.766469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911af5a-5141-4a46-a1fd-86e34cfe6207 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:55.767965 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911af5a-5141-4a46-a1fd-86e34cfe6207 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:55.774983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 21/41] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:46:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:46:55.774983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911af5a-5141-4a46-a1fd-86e34cfe6207 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:55.782244 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 21/42] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:46:55.783080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4058ba-056b-49e9-99c4-71aa420b8f1b None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:55.849278 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:55.880619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911af5a-5141-4a46-a1fd-86e34cfe6207 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:55.889621 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4058ba-056b-49e9-99c4-71aa420b8f1b None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:55.894637 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:55.917929 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbec84-0631-421f-979c-85ca7ff3078f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:55.918639 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbec84-0631-421f-979c-85ca7ff3078f None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:55.918933 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbec84-0631-421f-979c-85ca7ff3078f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:55.919219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cbec84-0631-421f-979c-85ca7ff3078f None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:55.919963 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbec84-0631-421f-979c-85ca7ff3078f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:55.921471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cbec84-0631-421f-979c-85ca7ff3078f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:55.932176 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a143ba-aa2b-407b-b9cb-32e04f805fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:55.933057 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a143ba-aa2b-407b-b9cb-32e04f805fd2 None admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:55.933346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a143ba-aa2b-407b-b9cb-32e04f805fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:55.933634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a143ba-aa2b-407b-b9cb-32e04f805fd2 None admin] PATH_INFO: `/v3/domains/default/users/1288d3a7956c42f7ad18e2e126c7dd07/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:55.935230 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 22/43] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jun 16 22:46:55 2022] POST /identity/v3/roles => generated 223 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:46:55.944512 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a143ba-aa2b-407b-b9cb-32e04f805fd2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1288d3a7956c42f7ad18e2e126c7dd07, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:55.944608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbec84-0631-421f-979c-85ca7ff3078f None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:55.944752 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a143ba-aa2b-407b-b9cb-32e04f805fd2 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:55.954784 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 22/44] 10.222.0.17 () {58 vars in 1361 bytes} [Thu Jun 16 22:46:55 2022] PUT /identity/v3/domains/default/users/1288d3a7956c42f7ad18e2e126c7dd07/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:46:56.073835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cbec84-0631-421f-979c-85ca7ff3078f None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:56.114658 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e8bd0-dc3e-4c85-a507-e06e69c017f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:56.115767 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e8bd0-dc3e-4c85-a507-e06e69c017f8 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:56.116061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e8bd0-dc3e-4c85-a507-e06e69c017f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:56.116334 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e8bd0-dc3e-4c85-a507-e06e69c017f8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:56.117584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e8bd0-dc3e-4c85-a507-e06e69c017f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:56.119079 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e8bd0-dc3e-4c85-a507-e06e69c017f8 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:56.120918 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 23/45] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:46:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:46:56.127025 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a143ba-aa2b-407b-b9cb-32e04f805fd2 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:56.239518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a143ba-aa2b-407b-b9cb-32e04f805fd2 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:56.269069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:56.278201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483da87-86a2-4cdc-aac1-f388e76612c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:56.278911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483da87-86a2-4cdc-aac1-f388e76612c6 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:56.279205 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483da87-86a2-4cdc-aac1-f388e76612c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:56.279489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483da87-86a2-4cdc-aac1-f388e76612c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:56.280210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483da87-86a2-4cdc-aac1-f388e76612c6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:56.281669 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483da87-86a2-4cdc-aac1-f388e76612c6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:56.307907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 23/46] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 22:46:56 2022] POST /identity/v3/projects => generated 312 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:46:58.097738 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5112e9-5577-4ac0-bbfa-7d5023d163d5 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:58.098251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5112e9-5577-4ac0-bbfa-7d5023d163d5 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:58.098337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5112e9-5577-4ac0-bbfa-7d5023d163d5 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:58.099830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 24/47] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:46:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:46:58.104914 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c436d55-949b-4d7f-b93e-8a973badcc1d None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:58.105287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c436d55-949b-4d7f-b93e-8a973badcc1d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:58.105598 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c436d55-949b-4d7f-b93e-8a973badcc1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:58.122694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-6c436d55-949b-4d7f-b93e-8a973badcc1d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:46:58.123868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-6c436d55-949b-4d7f-b93e-8a973badcc1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:46:58.188831 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 24/48] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:46:58.211792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5112e9-5577-4ac0-bbfa-7d5023d163d5 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:58.257819 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:58.317137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5112e9-5577-4ac0-bbfa-7d5023d163d5 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:58.351655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171fc50-adba-4984-90ab-a66c16426f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:58.352405 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171fc50-adba-4984-90ab-a66c16426f75 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:58.352697 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171fc50-adba-4984-90ab-a66c16426f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:58.352981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171fc50-adba-4984-90ab-a66c16426f75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:58.364961 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9171fc50-adba-4984-90ab-a66c16426f75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:46:58.372916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-9171fc50-adba-4984-90ab-a66c16426f75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:46:58.372916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:46:58.372916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:46:58.372916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:46:58.372916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:46:58.372916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:46:58.372916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:46:58.372916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:46:58.372916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:46:58.372916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:46:58.372916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:46:58.372916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:46:58.372916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:46:58.372916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:46:58.372916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:46:58.372916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:46:58.373992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:46:58.373992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:46:58.373992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:46:58.373992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:46:58.373992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:46:58.373992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:46:58.373992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:46:58.373992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:46:58.373992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:46:58.373992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:46:58.373992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:46:58.373992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:46:58.373992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:46:58.373992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:46:58.373992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:46:58.373992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:46:58.375197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:46:58.375197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:46:58.375197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:46:58.375197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:46:58.375197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:46:58.375197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:46:58.375197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:46:58.375197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:46:58.375197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:46:58.375197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:46:58.375197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:46:58.375197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:46:58.375197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 25/49] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:46:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:46:58.380724 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c436d55-949b-4d7f-b93e-8a973badcc1d None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:58.416986 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a702aa-47c9-4b5c-8f18-c43a02489729 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:58.417284 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a702aa-47c9-4b5c-8f18-c43a02489729 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:58.417569 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a702aa-47c9-4b5c-8f18-c43a02489729 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:58.419585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 26/50] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:46:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:46:58.424479 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f02fe-b6df-44c4-8140-41c43fedb263 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:58.424771 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f02fe-b6df-44c4-8140-41c43fedb263 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:58.425080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54f02fe-b6df-44c4-8140-41c43fedb263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:58.441237 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-d54f02fe-b6df-44c4-8140-41c43fedb263 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:46:58.442802 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d54f02fe-b6df-44c4-8140-41c43fedb263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:46:58.483119 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c436d55-949b-4d7f-b93e-8a973badcc1d None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:58.494224 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:58.512480 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 27/51] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:46:58.520091 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef4dbb-1af3-45aa-a49a-8ea137e05d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:58.520766 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef4dbb-1af3-45aa-a49a-8ea137e05d64 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:58.521057 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef4dbb-1af3-45aa-a49a-8ea137e05d64 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:58.521338 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ef4dbb-1af3-45aa-a49a-8ea137e05d64 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:58.522094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef4dbb-1af3-45aa-a49a-8ea137e05d64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:46:58.522405 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef4dbb-1af3-45aa-a49a-8ea137e05d64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:58.524081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ef4dbb-1af3-45aa-a49a-8ea137e05d64 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:58.535518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 25/52] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:46:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:46:58.536629 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f02fe-b6df-44c4-8140-41c43fedb263 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:58.540902 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef4dbb-1af3-45aa-a49a-8ea137e05d64 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:58.567373 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:58.635742 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54f02fe-b6df-44c4-8140-41c43fedb263 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:58.637516 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ef4dbb-1af3-45aa-a49a-8ea137e05d64 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:58.668913 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff9718-85cc-452e-99cf-fb795efa5a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:58.669707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff9718-85cc-452e-99cf-fb795efa5a36 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:58.669997 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff9718-85cc-452e-99cf-fb795efa5a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:58.670170 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49cc26-77dd-4ff0-8c20-c8121052f527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:58.670269 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff9718-85cc-452e-99cf-fb795efa5a36 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:58.671069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49cc26-77dd-4ff0-8c20-c8121052f527 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:58.671356 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49cc26-77dd-4ff0-8c20-c8121052f527 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:58.671530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff9718-85cc-452e-99cf-fb795efa5a36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:58.671635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49cc26-77dd-4ff0-8c20-c8121052f527 None admin] PATH_INFO: `/v3/users/1288d3a7956c42f7ad18e2e126c7dd07` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:58.672842 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49cc26-77dd-4ff0-8c20-c8121052f527 None admin] RBAC: Authorizing `identity:get_user(user_id=1288d3a7956c42f7ad18e2e126c7dd07)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:58.672978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff9718-85cc-452e-99cf-fb795efa5a36 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:58.674281 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff49cc26-77dd-4ff0-8c20-c8121052f527 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:58.674633 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 28/53] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:46:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:46:58.678630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 26/54] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:46:58 2022] GET /identity/v3/users/1288d3a7956c42f7ad18e2e126c7dd07 => generated 247 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:46:58.680208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff9718-85cc-452e-99cf-fb795efa5a36 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:58.681062 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49cc26-77dd-4ff0-8c20-c8121052f527 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:58.722782 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:58.777139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:58.780170 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49cc26-77dd-4ff0-8c20-c8121052f527 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:58.780388 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff9718-85cc-452e-99cf-fb795efa5a36 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:58.812779 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31360c0e-f232-4e63-8a09-8be52c7cf813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:58.813567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31360c0e-f232-4e63-8a09-8be52c7cf813 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:58.813856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31360c0e-f232-4e63-8a09-8be52c7cf813 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:58.813946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04e679-884b-4e31-a446-b8ee7d38a7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:58.814137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31360c0e-f232-4e63-8a09-8be52c7cf813 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:58.814513 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04e679-884b-4e31-a446-b8ee7d38a7ae None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:58.814836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04e679-884b-4e31-a446-b8ee7d38a7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:58.815121 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04e679-884b-4e31-a446-b8ee7d38a7ae None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:58.815378 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31360c0e-f232-4e63-8a09-8be52c7cf813 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:58.816348 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04e679-884b-4e31-a446-b8ee7d38a7ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:58.817365 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31360c0e-f232-4e63-8a09-8be52c7cf813 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:58.817786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04e679-884b-4e31-a446-b8ee7d38a7ae None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:58.819009 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 27/55] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:46:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:46:58.824619 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31360c0e-f232-4e63-8a09-8be52c7cf813 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:46:58.846073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 29/56] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jun 16 22:46:58 2022] POST /identity/v3/projects => generated 298 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 22:46:58.932356 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31360c0e-f232-4e63-8a09-8be52c7cf813 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:46:58.942704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:46:58.973057 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa75dc4d-889d-4b48-9f60-9949be84dee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:46:58.973745 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75dc4d-889d-4b48-9f60-9949be84dee3 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:46:58.974038 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75dc4d-889d-4b48-9f60-9949be84dee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:46:58.974352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa75dc4d-889d-4b48-9f60-9949be84dee3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:46:58.975274 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75dc4d-889d-4b48-9f60-9949be84dee3 None admin] RBAC: Adding query filter params (role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.domain.id=default, user.id=1288d3a7956c42f7ad18e2e126c7dd07) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:46:58.975594 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75dc4d-889d-4b48-9f60-9949be84dee3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:46:58.977090 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa75dc4d-889d-4b48-9f60-9949be84dee3 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:46:58.986396 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 28/57] 10.222.0.17 () {56 vars in 1322 bytes} [Thu Jun 16 22:46:58 2022] GET /identity/v3/role_assignments?user.id=1288d3a7956c42f7ad18e2e126c7dd07&scope.domain.id=default&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 530 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:00.961052 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629de9a-6778-4437-b398-eda8ff45eaa5 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:00.961561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629de9a-6778-4437-b398-eda8ff45eaa5 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:00.961642 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a629de9a-6778-4437-b398-eda8ff45eaa5 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:00.963061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 30/58] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:00.968128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4b891-4d1c-443a-8b2a-9c081a5b4352 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:00.968516 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4b891-4d1c-443a-8b2a-9c081a5b4352 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:00.968695 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f4b891-4d1c-443a-8b2a-9c081a5b4352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:00.985367 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-42f4b891-4d1c-443a-8b2a-9c081a5b4352 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:00.986552 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-42f4b891-4d1c-443a-8b2a-9c081a5b4352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:01.050220 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 29/59] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:47:01.074411 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629de9a-6778-4437-b398-eda8ff45eaa5 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:01.172249 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a629de9a-6778-4437-b398-eda8ff45eaa5 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:01.203926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51257dd7-b0b9-4c29-8963-7079f7056a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:01.204692 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51257dd7-b0b9-4c29-8963-7079f7056a4e None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:01.204978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51257dd7-b0b9-4c29-8963-7079f7056a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:01.205258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51257dd7-b0b9-4c29-8963-7079f7056a4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:01.206435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51257dd7-b0b9-4c29-8963-7079f7056a4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:01.208449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51257dd7-b0b9-4c29-8963-7079f7056a4e None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:01.210052 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 31/60] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:47:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:01.215418 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4b891-4d1c-443a-8b2a-9c081a5b4352 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:01.311872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f4b891-4d1c-443a-8b2a-9c081a5b4352 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:01.313356 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:01.346016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3af1f7-5ec1-4f9e-a79c-6bad62d483c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:01.346774 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3af1f7-5ec1-4f9e-a79c-6bad62d483c6 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:01.347068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3af1f7-5ec1-4f9e-a79c-6bad62d483c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:01.347371 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3af1f7-5ec1-4f9e-a79c-6bad62d483c6 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:01.348090 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3af1f7-5ec1-4f9e-a79c-6bad62d483c6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:01.349534 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3af1f7-5ec1-4f9e-a79c-6bad62d483c6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:01.373792 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 30/61] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 22:47:01 2022] POST /identity/v3/users => generated 275 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:47:04.514068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6582cd-5d02-467f-a7ea-4d065e925b0e None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:04.515084 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6582cd-5d02-467f-a7ea-4d065e925b0e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:04.515573 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6582cd-5d02-467f-a7ea-4d065e925b0e None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:04.517151 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 32/62] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:04.523127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f49bcf-54c9-40c4-8700-6f191030a9dd None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:04.524395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f49bcf-54c9-40c4-8700-6f191030a9dd None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:04.525441 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f49bcf-54c9-40c4-8700-6f191030a9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:04.551661 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-51f49bcf-54c9-40c4-8700-6f191030a9dd None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:04.553221 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-51f49bcf-54c9-40c4-8700-6f191030a9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:04.645086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 31/63] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:04.675548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6582cd-5d02-467f-a7ea-4d065e925b0e None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:04.686009 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdeed2-4796-4db1-ab18-436982280e10 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:04.688382 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdeed2-4796-4db1-ab18-436982280e10 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:04.688875 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdeed2-4796-4db1-ab18-436982280e10 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:04.690509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 32/64] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:04 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:04.700011 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06414b7f-ee3b-4fb9-b99f-765b99df9920 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:04.700527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06414b7f-ee3b-4fb9-b99f-765b99df9920 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:04.700978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06414b7f-ee3b-4fb9-b99f-765b99df9920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:04.705989 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:04.718200 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-06414b7f-ee3b-4fb9-b99f-765b99df9920 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:04.719756 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-06414b7f-ee3b-4fb9-b99f-765b99df9920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:04.832157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 33/65] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:04.855434 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06414b7f-ee3b-4fb9-b99f-765b99df9920 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:04.863279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6582cd-5d02-467f-a7ea-4d065e925b0e None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:04.880697 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:04.929860 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb384ec9-6fdc-4258-a540-de2505784d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:04.931159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb384ec9-6fdc-4258-a540-de2505784d6f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:04.933015 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb384ec9-6fdc-4258-a540-de2505784d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:04.934214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb384ec9-6fdc-4258-a540-de2505784d6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:04.960530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb384ec9-6fdc-4258-a540-de2505784d6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:04.973559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-cb384ec9-6fdc-4258-a540-de2505784d6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:04.973559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:04.973559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:04.973559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:04.973559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:04.973559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:04.973559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:04.973559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:04.973559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:04.973559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:04.973559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:04.973559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:04.973559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:04.973559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:04.973559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:04.973559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:04.974652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:04.974652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:04.974652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:04.974652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:04.974652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:04.974652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:04.974652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:04.974652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:04.974652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:04.974652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:04.974652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:04.974652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:04.974652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:04.974652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:04.974652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:04.974652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:04.975488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:04.975488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:04.975488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:04.975488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:04.975488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:04.975488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:04.975488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:04.975488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:04.975488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:04.975488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:04.975488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:04.975488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:04.980002 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 33/66] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:04 2022] GET /identity/v3/roles/member => generated 84 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:04.985852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb384ec9-6fdc-4258-a540-de2505784d6f None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:05.003985 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06414b7f-ee3b-4fb9-b99f-765b99df9920 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:05.062952 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040915f6-dcba-456c-8804-2a12e96586a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:05.064644 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040915f6-dcba-456c-8804-2a12e96586a6 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:05.065132 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040915f6-dcba-456c-8804-2a12e96586a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:05.065921 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040915f6-dcba-456c-8804-2a12e96586a6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:05.079624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040915f6-dcba-456c-8804-2a12e96586a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:05.087826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-040915f6-dcba-456c-8804-2a12e96586a6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:47:05.087826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:05.087826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:05.087826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:05.087826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:05.087826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:05.087826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:05.087826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:05.087826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:05.087826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:05.087826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:05.087826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:05.087826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:05.087826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:05.087826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:05.087826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:05.088627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:05.088627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:05.088627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:05.088627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:05.088627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:05.088627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:05.088627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:05.088627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:05.088627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:05.088627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:05.088627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:05.088627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:05.088627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:05.088627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:05.088627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:05.088627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:05.089426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:05.089426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:05.089426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:05.089426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:05.089426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:05.089426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:05.089426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:05.089426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:05.089426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:05.089426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:05.089426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:47:05.089426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:05.091351 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 34/67] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:47:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:47:05.092784 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df0259-5cfa-4bab-9c4c-e7ad9551489a None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:05.093225 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df0259-5cfa-4bab-9c4c-e7ad9551489a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:05.093628 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df0259-5cfa-4bab-9c4c-e7ad9551489a None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:05.095051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 35/68] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:05.096332 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2b1b5-822e-43a7-a1c5-8cc31aea4e35 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:05.096743 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2b1b5-822e-43a7-a1c5-8cc31aea4e35 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:05.097162 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a2b1b5-822e-43a7-a1c5-8cc31aea4e35 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:05.098575 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 36/69] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:05.099764 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:05.100530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df9779-f431-445a-8b97-7296a0c59096 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:05.100949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df9779-f431-445a-8b97-7296a0c59096 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:05.101648 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df9779-f431-445a-8b97-7296a0c59096 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:05.118923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-e4df9779-f431-445a-8b97-7296a0c59096 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:05.120077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb384ec9-6fdc-4258-a540-de2505784d6f None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:05.120181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e4df9779-f431-445a-8b97-7296a0c59096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:05.155851 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297495f5-af9f-4768-8c8c-563bb0fc49ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:05.156542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297495f5-af9f-4768-8c8c-563bb0fc49ec None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:05.156830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297495f5-af9f-4768-8c8c-563bb0fc49ec None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:05.157119 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297495f5-af9f-4768-8c8c-563bb0fc49ec None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:05.157873 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297495f5-af9f-4768-8c8c-563bb0fc49ec None admin] RBAC: Adding query filter params (name=member) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:05.158190 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297495f5-af9f-4768-8c8c-563bb0fc49ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:05.159599 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297495f5-af9f-4768-8c8c-563bb0fc49ec None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:05.170629 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 34/70] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:05.172120 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297495f5-af9f-4768-8c8c-563bb0fc49ec None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:05.202636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 37/71] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:47:05.202636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa73e07-1d67-4e52-b256-d599b55585ba None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:05.202636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa73e07-1d67-4e52-b256-d599b55585ba None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:05.202636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa73e07-1d67-4e52-b256-d599b55585ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:05.220935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-9aa73e07-1d67-4e52-b256-d599b55585ba None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:05.222687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-9aa73e07-1d67-4e52-b256-d599b55585ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:05.225667 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:05.292421 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 38/72] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:05.293263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa73e07-1d67-4e52-b256-d599b55585ba None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:05.301779 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297495f5-af9f-4768-8c8c-563bb0fc49ec None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:05.334946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f67ce3-0b06-4ed1-bc08-868b71e2f321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:05.335623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f67ce3-0b06-4ed1-bc08-868b71e2f321 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:05.335910 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f67ce3-0b06-4ed1-bc08-868b71e2f321 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:05.336187 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f67ce3-0b06-4ed1-bc08-868b71e2f321 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:05.336927 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f67ce3-0b06-4ed1-bc08-868b71e2f321 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:05.337240 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f67ce3-0b06-4ed1-bc08-868b71e2f321 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:05.338694 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f67ce3-0b06-4ed1-bc08-868b71e2f321 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:05.346663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 35/73] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:05.347316 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f67ce3-0b06-4ed1-bc08-868b71e2f321 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:05.354812 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:05.398188 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa73e07-1d67-4e52-b256-d599b55585ba None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:05.434940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f986d0-13d4-42aa-af3b-92dfb05bc073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:05.435831 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f986d0-13d4-42aa-af3b-92dfb05bc073 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:05.436124 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f986d0-13d4-42aa-af3b-92dfb05bc073 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:05.436406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f986d0-13d4-42aa-af3b-92dfb05bc073 None admin] PATH_INFO: `/v3/users/506c986a39184b05b22b043ea6c2f174` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:05.443901 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:05.452455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f986d0-13d4-42aa-af3b-92dfb05bc073 None admin] RBAC: Authorizing `identity:get_user(user_id=506c986a39184b05b22b043ea6c2f174)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:05.453913 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f986d0-13d4-42aa-af3b-92dfb05bc073 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:05.455688 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 39/74] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:05 2022] GET /identity/v3/users/506c986a39184b05b22b043ea6c2f174 => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:05.456894 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183126d3-f80a-47f1-941d-3c6babbb4de6 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:05.457191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183126d3-f80a-47f1-941d-3c6babbb4de6 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:05.457480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183126d3-f80a-47f1-941d-3c6babbb4de6 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:05.458786 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 40/75] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:05.459492 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183126d3-f80a-47f1-941d-3c6babbb4de6 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:05.463417 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f67ce3-0b06-4ed1-bc08-868b71e2f321 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:05.497882 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef970e09-f1e8-4231-b070-6f20f7608b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:05.498712 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef970e09-f1e8-4231-b070-6f20f7608b06 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:05.499006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef970e09-f1e8-4231-b070-6f20f7608b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:05.499287 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef970e09-f1e8-4231-b070-6f20f7608b06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:05.500472 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef970e09-f1e8-4231-b070-6f20f7608b06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:05.501941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef970e09-f1e8-4231-b070-6f20f7608b06 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:05.503783 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 36/76] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:47:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:05.504391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef970e09-f1e8-4231-b070-6f20f7608b06 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:05.560789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183126d3-f80a-47f1-941d-3c6babbb4de6 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:05.574375 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:05.594825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175b6f4-6b97-407e-9e54-646278648723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:05.595583 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175b6f4-6b97-407e-9e54-646278648723 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:05.595877 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175b6f4-6b97-407e-9e54-646278648723 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:05.596162 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175b6f4-6b97-407e-9e54-646278648723 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:05.605069 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef970e09-f1e8-4231-b070-6f20f7608b06 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:05.606155 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175b6f4-6b97-407e-9e54-646278648723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:05.614193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-c175b6f4-6b97-407e-9e54-646278648723 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:05.614193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:05.614193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:05.614193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:05.614193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:05.614193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:05.614193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:05.614193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:05.614193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:05.614193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:05.614193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:05.614193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:05.614193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:05.614193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:05.614193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:05.614193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:05.615186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:05.615186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:05.615186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:05.615186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:05.615186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:05.615186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:05.615186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:05.615186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:05.615186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:05.615186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:05.615186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:05.615186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:05.615186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:05.615186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:05.615186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:05.615186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:05.615957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:05.615957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:05.615957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:05.615957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:05.615957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:05.615957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:05.615957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:05.615957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:05.615957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:05.615957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:05.615957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:05.615957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:05.615957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 41/77] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:47:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:05.616595 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175b6f4-6b97-407e-9e54-646278648723 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:05.638875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf101df-233b-43f1-b265-ec9563969bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:05.639761 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf101df-233b-43f1-b265-ec9563969bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:05.640056 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf101df-233b-43f1-b265-ec9563969bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:05.640347 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf101df-233b-43f1-b265-ec9563969bb6 None admin] PATH_INFO: `/v3/users/1288d3a7956c42f7ad18e2e126c7dd07` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:05.641557 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf101df-233b-43f1-b265-ec9563969bb6 None admin] RBAC: Authorizing `identity:get_user(user_id=1288d3a7956c42f7ad18e2e126c7dd07)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:05.643060 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf101df-233b-43f1-b265-ec9563969bb6 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:05.644702 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 37/78] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:05 2022] GET /identity/v3/users/1288d3a7956c42f7ad18e2e126c7dd07 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:05.646508 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2180c82-9f74-4abf-9103-91cb3b332d56 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:05.647072 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2180c82-9f74-4abf-9103-91cb3b332d56 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:05.647529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2180c82-9f74-4abf-9103-91cb3b332d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:05.665143 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-a2180c82-9f74-4abf-9103-91cb3b332d56 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:05.666359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-a2180c82-9f74-4abf-9103-91cb3b332d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:05.674834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:05.720169 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175b6f4-6b97-407e-9e54-646278648723 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:05.734714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 38/79] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:05.735537 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2180c82-9f74-4abf-9103-91cb3b332d56 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:05.761299 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2205b0c3-5331-47a1-b448-6762cf1c3b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:05.761959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205b0c3-5331-47a1-b448-6762cf1c3b52 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:05.762254 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205b0c3-5331-47a1-b448-6762cf1c3b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:05.762544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2205b0c3-5331-47a1-b448-6762cf1c3b52 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:05.803492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:05.833407 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2205b0c3-5331-47a1-b448-6762cf1c3b52 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:05.833788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2180c82-9f74-4abf-9103-91cb3b332d56 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:05.834906 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2205b0c3-5331-47a1-b448-6762cf1c3b52 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:05.868415 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712931b-3bd5-4f50-91f8-dcf2d3ff0eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:05.869093 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712931b-3bd5-4f50-91f8-dcf2d3ff0eab None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:05.869380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712931b-3bd5-4f50-91f8-dcf2d3ff0eab None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:05.869659 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712931b-3bd5-4f50-91f8-dcf2d3ff0eab None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:05.870369 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a712931b-3bd5-4f50-91f8-dcf2d3ff0eab None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:05.871863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a712931b-3bd5-4f50-91f8-dcf2d3ff0eab None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:05.889192 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:05.910628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 39/80] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 22:47:05 2022] POST /identity/v3/users => generated 289 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:47:05.910628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712931b-3bd5-4f50-91f8-dcf2d3ff0eab None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:05.940230 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 42/81] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:05.941171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2205b0c3-5331-47a1-b448-6762cf1c3b52 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:06.029517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712931b-3bd5-4f50-91f8-dcf2d3ff0eab None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:06.055272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2205b0c3-5331-47a1-b448-6762cf1c3b52 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:06.060673 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:06.078107 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73988d9f-f1e5-4ea3-82ad-90e7e4afece5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:06.078836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73988d9f-f1e5-4ea3-82ad-90e7e4afece5 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:06.079125 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73988d9f-f1e5-4ea3-82ad-90e7e4afece5 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:06.079406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73988d9f-f1e5-4ea3-82ad-90e7e4afece5 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:06.080168 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73988d9f-f1e5-4ea3-82ad-90e7e4afece5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:06.080478 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73988d9f-f1e5-4ea3-82ad-90e7e4afece5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:06.081894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73988d9f-f1e5-4ea3-82ad-90e7e4afece5 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:06.091685 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 40/82] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:47:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:06.092510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73988d9f-f1e5-4ea3-82ad-90e7e4afece5 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:06.106113 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4f44a-083e-4366-b1eb-a3dd62751315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:06.106820 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4f44a-083e-4366-b1eb-a3dd62751315 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:06.107114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4f44a-083e-4366-b1eb-a3dd62751315 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:06.107398 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4f44a-083e-4366-b1eb-a3dd62751315 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:06.135453 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:06.187644 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4f44a-083e-4366-b1eb-a3dd62751315 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:06.189254 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb4f44a-083e-4366-b1eb-a3dd62751315 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:06.222856 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73988d9f-f1e5-4ea3-82ad-90e7e4afece5 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:06.273044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538b92a-275c-4ebe-9047-605be6eb491b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:06.274375 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538b92a-275c-4ebe-9047-605be6eb491b None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:06.275318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538b92a-275c-4ebe-9047-605be6eb491b None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:06.275768 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538b92a-275c-4ebe-9047-605be6eb491b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:06.291273 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e538b92a-275c-4ebe-9047-605be6eb491b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:06.301773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-e538b92a-275c-4ebe-9047-605be6eb491b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:06.301773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:06.301773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:06.301773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:06.301773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:06.301773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:06.301773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:06.301773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:06.301773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:06.301773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:06.301773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:06.301773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:06.301773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:06.301773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:06.301773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:06.301773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:06.302861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:06.302861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:06.302861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:06.302861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:06.302861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:06.302861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:06.302861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:06.302861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:06.302861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:06.302861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:06.302861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:06.302861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:06.302861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:06.302861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:06.302861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:06.302861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:06.303778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:06.303778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:06.303778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:06.303778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:06.303778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:06.303778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:06.303778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:06.303778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:06.303778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:06.303778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:06.303778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:06.303778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:06.306956 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 41/83] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:06.307979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538b92a-275c-4ebe-9047-605be6eb491b None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:06.322227 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 43/84] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:06.327693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4f44a-083e-4366-b1eb-a3dd62751315 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:06.364445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:06.392575 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:06.424496 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538b92a-275c-4ebe-9047-605be6eb491b None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:06.442875 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4f44a-083e-4366-b1eb-a3dd62751315 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:06.460704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9494f-de2f-4df9-9a11-d5e1969dc534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:06.461788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9494f-de2f-4df9-9a11-d5e1969dc534 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:06.462241 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9494f-de2f-4df9-9a11-d5e1969dc534 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:06.462702 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed9494f-de2f-4df9-9a11-d5e1969dc534 None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:06.464052 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9494f-de2f-4df9-9a11-d5e1969dc534 None admin] RBAC: Authorizing `identity:get_project(project_id=9bdf56f0db064649a642e4d566966738)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:06.465656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed9494f-de2f-4df9-9a11-d5e1969dc534 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:06.467531 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 42/85] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:06 2022] GET /identity/v3/projects/9bdf56f0db064649a642e4d566966738 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:06.468432 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9494f-de2f-4df9-9a11-d5e1969dc534 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:06.481148 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4c638-fa5c-47b9-adfa-6ccaa0ab6558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:06.482042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4c638-fa5c-47b9-adfa-6ccaa0ab6558 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:06.482388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4c638-fa5c-47b9-adfa-6ccaa0ab6558 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:06.482627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a4c638-fa5c-47b9-adfa-6ccaa0ab6558 None admin] PATH_INFO: `/v3/users/506c986a39184b05b22b043ea6c2f174` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:06.483833 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4c638-fa5c-47b9-adfa-6ccaa0ab6558 None admin] RBAC: Authorizing `identity:get_user(user_id=506c986a39184b05b22b043ea6c2f174)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:06.485380 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a4c638-fa5c-47b9-adfa-6ccaa0ab6558 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:06.487219 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 44/86] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:06 2022] GET /identity/v3/users/506c986a39184b05b22b043ea6c2f174 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:06.493075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4c638-fa5c-47b9-adfa-6ccaa0ab6558 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:06.573871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed9494f-de2f-4df9-9a11-d5e1969dc534 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:06.594952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:06.595908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a4c638-fa5c-47b9-adfa-6ccaa0ab6558 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:06.610440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac66eeb-f243-4113-bde4-924abe61080c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:06.611453 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac66eeb-f243-4113-bde4-924abe61080c None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:06.611939 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac66eeb-f243-4113-bde4-924abe61080c None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:06.612414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac66eeb-f243-4113-bde4-924abe61080c None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:06.613378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac66eeb-f243-4113-bde4-924abe61080c None admin] RBAC: Adding query filter params (name=member) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:06.614434 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac66eeb-f243-4113-bde4-924abe61080c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:06.616107 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac66eeb-f243-4113-bde4-924abe61080c None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:06.625039 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 43/87] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:06.626125 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac66eeb-f243-4113-bde4-924abe61080c None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:06.627006 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:06.637121 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d0d59-9804-4607-9898-fb8f9f1ec332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:06.637995 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d0d59-9804-4607-9898-fb8f9f1ec332 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:06.638283 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d0d59-9804-4607-9898-fb8f9f1ec332 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:06.638565 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54d0d59-9804-4607-9898-fb8f9f1ec332 None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:06.640105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d0d59-9804-4607-9898-fb8f9f1ec332 None admin] RBAC: Authorizing `identity:get_project(project_id=9bdf56f0db064649a642e4d566966738)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:06.641448 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54d0d59-9804-4607-9898-fb8f9f1ec332 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:06.648870 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 45/88] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:06 2022] GET /identity/v3/projects/9bdf56f0db064649a642e4d566966738 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:06.649601 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d0d59-9804-4607-9898-fb8f9f1ec332 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:06.737531 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac66eeb-f243-4113-bde4-924abe61080c None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:06.766276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54d0d59-9804-4607-9898-fb8f9f1ec332 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:06.773096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cdf382-8651-4418-9060-f1632f45cb3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:06.774026 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdf382-8651-4418-9060-f1632f45cb3e None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:06.774510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdf382-8651-4418-9060-f1632f45cb3e None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:06.775296 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cdf382-8651-4418-9060-f1632f45cb3e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:06.776338 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cdf382-8651-4418-9060-f1632f45cb3e None admin] RBAC: Adding query filter params (role.id=bff4f694fb57451e9a0783dc014187f2, scope.project.id=9bdf56f0db064649a642e4d566966738, user.id=506c986a39184b05b22b043ea6c2f174) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:06.776863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cdf382-8651-4418-9060-f1632f45cb3e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:06.778506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cdf382-8651-4418-9060-f1632f45cb3e None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:06.784531 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 44/89] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:47:06 2022] GET /identity/v3/role_assignments?user.id=506c986a39184b05b22b043ea6c2f174&scope.project.id=9bdf56f0db064649a642e4d566966738&role.id=bff4f694fb57451e9a0783dc014187f2 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:06.786113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cdf382-8651-4418-9060-f1632f45cb3e None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:06.820207 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1a76f-d96d-488e-a47c-ebcacda3bdef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:06.820923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1a76f-d96d-488e-a47c-ebcacda3bdef None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:06.821227 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1a76f-d96d-488e-a47c-ebcacda3bdef None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:06.821515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe1a76f-d96d-488e-a47c-ebcacda3bdef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:06.835191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:06.899251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cdf382-8651-4418-9060-f1632f45cb3e None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:06.930763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:06.937906 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63e8c5-42f5-4438-85c1-0fea369e8f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:06.939271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63e8c5-42f5-4438-85c1-0fea369e8f69 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:06.939777 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63e8c5-42f5-4438-85c1-0fea369e8f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:06.940338 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc63e8c5-42f5-4438-85c1-0fea369e8f69 None admin] PATH_INFO: `/v3/users/506c986a39184b05b22b043ea6c2f174` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:06.942394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63e8c5-42f5-4438-85c1-0fea369e8f69 None admin] RBAC: Authorizing `identity:get_user(user_id=506c986a39184b05b22b043ea6c2f174)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:06.944111 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc63e8c5-42f5-4438-85c1-0fea369e8f69 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:06.946064 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 45/90] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:06 2022] GET /identity/v3/users/506c986a39184b05b22b043ea6c2f174 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:06.947137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63e8c5-42f5-4438-85c1-0fea369e8f69 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:06.957679 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1a76f-d96d-488e-a47c-ebcacda3bdef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:06.959576 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe1a76f-d96d-488e-a47c-ebcacda3bdef None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:07.118720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 46/91] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:07.119669 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1a76f-d96d-488e-a47c-ebcacda3bdef None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:07.204975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc63e8c5-42f5-4438-85c1-0fea369e8f69 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:07.230657 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:07.262548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:07.279563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe1a76f-d96d-488e-a47c-ebcacda3bdef None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:07.287145 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa1ace-0149-41b4-9f5e-c78b2ce838f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:07.287822 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa1ace-0149-41b4-9f5e-c78b2ce838f4 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:07.288110 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa1ace-0149-41b4-9f5e-c78b2ce838f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:07.288392 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23aa1ace-0149-41b4-9f5e-c78b2ce838f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:07.289227 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa1ace-0149-41b4-9f5e-c78b2ce838f4 None admin] RBAC: Adding query filter params (role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.project.id=9bdf56f0db064649a642e4d566966738, user.id=1288d3a7956c42f7ad18e2e126c7dd07) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:07.289544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa1ace-0149-41b4-9f5e-c78b2ce838f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:07.291030 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23aa1ace-0149-41b4-9f5e-c78b2ce838f4 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:07.305018 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 46/92] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:47:06 2022] GET /identity/v3/role_assignments?user.id=1288d3a7956c42f7ad18e2e126c7dd07&scope.project.id=9bdf56f0db064649a642e4d566966738&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 259 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:07.305936 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa1ace-0149-41b4-9f5e-c78b2ce838f4 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:07.416091 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc30351-044a-4930-8551-b24a1ec300a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:07.416785 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc30351-044a-4930-8551-b24a1ec300a7 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:07.417076 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc30351-044a-4930-8551-b24a1ec300a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:07.417366 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc30351-044a-4930-8551-b24a1ec300a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:07.452982 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23aa1ace-0149-41b4-9f5e-c78b2ce838f4 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:07.491583 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36399a35-2dc7-4e04-a285-5ce3d0dd6aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:07.492780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36399a35-2dc7-4e04-a285-5ce3d0dd6aef None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:07.493269 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36399a35-2dc7-4e04-a285-5ce3d0dd6aef None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:07.493766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36399a35-2dc7-4e04-a285-5ce3d0dd6aef None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:07.495288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36399a35-2dc7-4e04-a285-5ce3d0dd6aef None admin] RBAC: Authorizing `identity:get_project(project_id=9bdf56f0db064649a642e4d566966738)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:07.497032 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36399a35-2dc7-4e04-a285-5ce3d0dd6aef None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:07.499013 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 47/93] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:07 2022] GET /identity/v3/projects/9bdf56f0db064649a642e4d566966738 => generated 298 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:07.505137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36399a35-2dc7-4e04-a285-5ce3d0dd6aef None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:07.557196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:07.603763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc30351-044a-4930-8551-b24a1ec300a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:07.605263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc30351-044a-4930-8551-b24a1ec300a7 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:07.629034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36399a35-2dc7-4e04-a285-5ce3d0dd6aef None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:07.670831 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7c8af-b26d-4b5c-b0d6-650183ed4586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:07.671887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7c8af-b26d-4b5c-b0d6-650183ed4586 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:07.672421 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7c8af-b26d-4b5c-b0d6-650183ed4586 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:07.672894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7c8af-b26d-4b5c-b0d6-650183ed4586 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:07.673918 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7c8af-b26d-4b5c-b0d6-650183ed4586 None admin] RBAC: Adding query filter params (role.id=bda1c48ca2534bc6b2ca4f3ff479b143, scope.project.id=9bdf56f0db064649a642e4d566966738, user.id=506c986a39184b05b22b043ea6c2f174) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:07.674449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7c8af-b26d-4b5c-b0d6-650183ed4586 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:07.676485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab7c8af-b26d-4b5c-b0d6-650183ed4586 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:07.683288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 48/94] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:47:07 2022] GET /identity/v3/role_assignments?user.id=506c986a39184b05b22b043ea6c2f174&scope.project.id=9bdf56f0db064649a642e4d566966738&role.id=bda1c48ca2534bc6b2ca4f3ff479b143 => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:07.783065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:07.795964 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 47/95] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 676 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:07.801747 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7c8af-b26d-4b5c-b0d6-650183ed4586 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:07.923552 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab7c8af-b26d-4b5c-b0d6-650183ed4586 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:07.931939 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:07.962875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111e722d-512d-4053-bba4-11331a029d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:07.964129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e722d-512d-4053-bba4-11331a029d88 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:07.964614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e722d-512d-4053-bba4-11331a029d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:07.965088 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111e722d-512d-4053-bba4-11331a029d88 None admin] PATH_INFO: `/v3/projects/2ec770657a0e4414a0c09486f5252563` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:07.966528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e722d-512d-4053-bba4-11331a029d88 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec770657a0e4414a0c09486f5252563)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:07.968255 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111e722d-512d-4053-bba4-11331a029d88 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:07.970544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 49/96] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:07 2022] GET /identity/v3/projects/2ec770657a0e4414a0c09486f5252563 => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:07.976815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc30351-044a-4930-8551-b24a1ec300a7 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:08.123290 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc30351-044a-4930-8551-b24a1ec300a7 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:08.171887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90465986-c4db-4f7e-8c92-a2ade5f811d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:08.172570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90465986-c4db-4f7e-8c92-a2ade5f811d2 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:08.172863 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90465986-c4db-4f7e-8c92-a2ade5f811d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:08.173146 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90465986-c4db-4f7e-8c92-a2ade5f811d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:08.174001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90465986-c4db-4f7e-8c92-a2ade5f811d2 None admin] RBAC: Adding query filter params (role.id=bff4f694fb57451e9a0783dc014187f2, scope.project.id=2ec770657a0e4414a0c09486f5252563, user.id=506c986a39184b05b22b043ea6c2f174) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:08.174327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90465986-c4db-4f7e-8c92-a2ade5f811d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:08.175847 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90465986-c4db-4f7e-8c92-a2ade5f811d2 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:08.183156 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 48/97] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:47:07 2022] GET /identity/v3/role_assignments?user.id=506c986a39184b05b22b043ea6c2f174&scope.project.id=2ec770657a0e4414a0c09486f5252563&role.id=bff4f694fb57451e9a0783dc014187f2 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:08.459233 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a67ce8-1884-49e9-8bd9-e20c6dcd654b None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:08.459938 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a67ce8-1884-49e9-8bd9-e20c6dcd654b None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:08.460440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a67ce8-1884-49e9-8bd9-e20c6dcd654b None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:08.462026 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 50/98] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:08.467486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f08f0-8e85-4801-b469-f9c0d2bb18dc None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:08.467786 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f08f0-8e85-4801-b469-f9c0d2bb18dc None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:08.468077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4f08f0-8e85-4801-b469-f9c0d2bb18dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:08.491621 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-1a4f08f0-8e85-4801-b469-f9c0d2bb18dc None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:08.492835 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-1a4f08f0-8e85-4801-b469-f9c0d2bb18dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:08.566666 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:08.592372 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 49/99] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:08.634769 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a67ce8-1884-49e9-8bd9-e20c6dcd654b None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:08.635477 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76710762-c37d-4c21-9a9e-9a7507c0f1ab None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:08.635838 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76710762-c37d-4c21-9a9e-9a7507c0f1ab None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:08.636053 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76710762-c37d-4c21-9a9e-9a7507c0f1ab None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:08.637454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 50/100] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:47:08.646630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f0fe1-5d28-4a2b-8a0f-43e0ebe4af1b None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:08.646630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f0fe1-5d28-4a2b-8a0f-43e0ebe4af1b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:08.646630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f0fe1-5d28-4a2b-8a0f-43e0ebe4af1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:08.675663 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-7c1f0fe1-5d28-4a2b-8a0f-43e0ebe4af1b None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:08.677191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7c1f0fe1-5d28-4a2b-8a0f-43e0ebe4af1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:08.768982 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 51/101] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 16 22:47:08.814254 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f0fe1-5d28-4a2b-8a0f-43e0ebe4af1b None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:08.830618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a67ce8-1884-49e9-8bd9-e20c6dcd654b None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:08.919965 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bd4d5-7a98-4d26-bac3-bcad77ecc51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:08.920792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bd4d5-7a98-4d26-bac3-bcad77ecc51f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:08.921110 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bd4d5-7a98-4d26-bac3-bcad77ecc51f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:08.921357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072bd4d5-7a98-4d26-bac3-bcad77ecc51f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:08.941316 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072bd4d5-7a98-4d26-bac3-bcad77ecc51f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:08.957359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-072bd4d5-7a98-4d26-bac3-bcad77ecc51f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 22:47:08.957359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:08.957359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:08.957359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:08.957359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:08.957359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:08.957359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:08.957359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:08.957359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:08.957359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:08.957359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:08.957359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:08.957359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:08.957359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:08.957359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:08.957359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:08.958901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:08.958901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:08.958901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:08.958901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:08.958901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:08.958901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:08.958901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:08.958901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:08.958901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:08.958901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:08.958901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:08.958901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:08.958901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:08.958901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:08.958901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:08.958901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:08.959628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:08.959628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:08.959628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:08.959628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:08.959628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:08.959628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:08.959628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:08.959628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:08.959628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:08.959628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:08.959628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 22:47:08.959628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:08.962056 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 51/102] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:08 2022] GET /identity/v3/roles/reader => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:08.983694 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bd4d5-7a98-4d26-bac3-bcad77ecc51f None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:08.984229 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f0fe1-5d28-4a2b-8a0f-43e0ebe4af1b None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:09.006973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:09.033653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20a8e6-f552-4196-b621-bcd952a6c110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:09.034623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20a8e6-f552-4196-b621-bcd952a6c110 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:09.034924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20a8e6-f552-4196-b621-bcd952a6c110 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:09.035260 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e20a8e6-f552-4196-b621-bcd952a6c110 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:09.039743 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20a8e6-f552-4196-b621-bcd952a6c110 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:09.041433 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e20a8e6-f552-4196-b621-bcd952a6c110 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:09.043297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 52/103] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:47:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:09.048839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20a8e6-f552-4196-b621-bcd952a6c110 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:09.106945 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:09.258137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e20a8e6-f552-4196-b621-bcd952a6c110 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:09.273288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072bd4d5-7a98-4d26-bac3-bcad77ecc51f None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:09.342466 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4163a3-0955-47d2-833b-bf9977199ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:09.343094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4163a3-0955-47d2-833b-bf9977199ef0 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:09.343385 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4163a3-0955-47d2-833b-bf9977199ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:09.343667 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4163a3-0955-47d2-833b-bf9977199ef0 None admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:09.344397 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4163a3-0955-47d2-833b-bf9977199ef0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:09.345842 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4163a3-0955-47d2-833b-bf9977199ef0 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:09.355423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9d9e7-ed88-4e4d-8298-bd542a487798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:09.356110 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9d9e7-ed88-4e4d-8298-bd542a487798 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:09.356399 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9d9e7-ed88-4e4d-8298-bd542a487798 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:09.356680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea9d9e7-ed88-4e4d-8298-bd542a487798 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:09.357437 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9d9e7-ed88-4e4d-8298-bd542a487798 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:09.357747 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9d9e7-ed88-4e4d-8298-bd542a487798 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:09.359152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea9d9e7-ed88-4e4d-8298-bd542a487798 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:09.381814 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 52/104] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:08 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:09.387754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9d9e7-ed88-4e4d-8298-bd542a487798 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:09.396711 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 53/105] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 22:47:09 2022] POST /identity/v3/projects => generated 302 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 22:47:09.529266 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:09.535394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea9d9e7-ed88-4e4d-8298-bd542a487798 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:09.573433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1da37-5334-4c68-8449-24163fa1ee92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:09.574576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1da37-5334-4c68-8449-24163fa1ee92 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:09.575065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1da37-5334-4c68-8449-24163fa1ee92 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:09.575492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b1da37-5334-4c68-8449-24163fa1ee92 None admin] PATH_INFO: `/v3/users/1e872aac4d734b22b3ef7ea653686c96` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:09.579087 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7b42e-cf3a-4557-8f13-4adce0155ea9 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:09.579624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7b42e-cf3a-4557-8f13-4adce0155ea9 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:09.580081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c7b42e-cf3a-4557-8f13-4adce0155ea9 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:09.581653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 54/106] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:47:09.588200 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c190fa6-1049-4307-998f-7df47065601d None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:09.588781 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c190fa6-1049-4307-998f-7df47065601d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:09.589326 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c190fa6-1049-4307-998f-7df47065601d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:09.594708 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1da37-5334-4c68-8449-24163fa1ee92 None admin] RBAC: Authorizing `identity:get_user(user_id=1e872aac4d734b22b3ef7ea653686c96)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:09.594708 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b1da37-5334-4c68-8449-24163fa1ee92 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:09.595525 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 53/107] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:09 2022] GET /identity/v3/users/1e872aac4d734b22b3ef7ea653686c96 => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:09.602336 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1da37-5334-4c68-8449-24163fa1ee92 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:09.633274 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-0c190fa6-1049-4307-998f-7df47065601d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:09.634783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-0c190fa6-1049-4307-998f-7df47065601d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:09.690970 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:09.716094 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b1da37-5334-4c68-8449-24163fa1ee92 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:09.752782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd91b73-e8fa-4a9d-8344-a50b2992082d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:09.753741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd91b73-e8fa-4a9d-8344-a50b2992082d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:09.754178 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd91b73-e8fa-4a9d-8344-a50b2992082d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:09.755191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd91b73-e8fa-4a9d-8344-a50b2992082d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:09.757390 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 55/108] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:09.783019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:09.808823 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c190fa6-1049-4307-998f-7df47065601d None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:09.829528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd91b73-e8fa-4a9d-8344-a50b2992082d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:09.831313 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd91b73-e8fa-4a9d-8344-a50b2992082d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:09.937118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 54/109] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 22:47:09.943064 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd91b73-e8fa-4a9d-8344-a50b2992082d None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:09.985688 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c190fa6-1049-4307-998f-7df47065601d None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:10.003919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:10.042241 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f7ad5-ef06-446e-9422-8a14959b0f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:10.043179 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f7ad5-ef06-446e-9422-8a14959b0f7a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:10.043476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f7ad5-ef06-446e-9422-8a14959b0f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:10.043760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581f7ad5-ef06-446e-9422-8a14959b0f7a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:10.049390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd91b73-e8fa-4a9d-8344-a50b2992082d None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:10.060235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581f7ad5-ef06-446e-9422-8a14959b0f7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:10.076509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-581f7ad5-ef06-446e-9422-8a14959b0f7a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:10.076509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:10.076509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:10.076509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:10.076509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:10.076509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:10.076509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:10.076509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:10.076509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:10.076509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:10.076509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:10.076509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:10.076509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:10.076509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:10.076509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:10.076509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:10.077542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:10.077542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:10.077542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:10.077542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:10.077542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:10.077542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:10.077542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:10.077542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:10.077542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:10.077542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:10.077542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:10.077542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:10.077542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:10.077542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:10.077542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:10.077542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:10.078437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:10.078437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:10.078437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:10.078437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:10.078437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:10.078437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:10.078437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:10.078437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:10.078437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:10.078437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:10.078437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:10.078437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:10.078437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 56/110] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:47:10.085027 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f7ad5-ef06-446e-9422-8a14959b0f7a None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:10.088967 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1bd83-1a06-4429-a55c-f557cafe483d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:10.090031 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1bd83-1a06-4429-a55c-f557cafe483d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:10.090473 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1bd83-1a06-4429-a55c-f557cafe483d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:10.090947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c1bd83-1a06-4429-a55c-f557cafe483d None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:10.092340 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1bd83-1a06-4429-a55c-f557cafe483d None admin] RBAC: Authorizing `identity:get_project(project_id=9bdf56f0db064649a642e4d566966738)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:10.094005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c1bd83-1a06-4429-a55c-f557cafe483d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:10.095900 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 55/111] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:09 2022] GET /identity/v3/projects/9bdf56f0db064649a642e4d566966738 => generated 298 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:10.102352 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1bd83-1a06-4429-a55c-f557cafe483d None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:10.129493 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:10.222721 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c1bd83-1a06-4429-a55c-f557cafe483d None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:10.222891 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581f7ad5-ef06-446e-9422-8a14959b0f7a None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:10.258197 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bde4a-843f-4073-9455-93b8256b9f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:10.258904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bde4a-843f-4073-9455-93b8256b9f13 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:10.260226 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bde4a-843f-4073-9455-93b8256b9f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:10.260515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6bde4a-843f-4073-9455-93b8256b9f13 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:10.261270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bde4a-843f-4073-9455-93b8256b9f13 None admin] RBAC: Adding query filter params (name=member) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:10.261583 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bde4a-843f-4073-9455-93b8256b9f13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:10.262974 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6bde4a-843f-4073-9455-93b8256b9f13 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:10.265591 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:10.280856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 57/112] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:10.287566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bde4a-843f-4073-9455-93b8256b9f13 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:10.292166 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9227d8-da56-4654-816a-6786c02c5df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:10.293040 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9227d8-da56-4654-816a-6786c02c5df0 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:10.293481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9227d8-da56-4654-816a-6786c02c5df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:10.293898 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9227d8-da56-4654-816a-6786c02c5df0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:10.294907 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9227d8-da56-4654-816a-6786c02c5df0 None admin] RBAC: Adding query filter params (role.id=8eefae9314f544de97883fa2b10a2046, scope.project.id=9bdf56f0db064649a642e4d566966738, user.id=1e872aac4d734b22b3ef7ea653686c96) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:10.295384 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9227d8-da56-4654-816a-6786c02c5df0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:10.296970 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9227d8-da56-4654-816a-6786c02c5df0 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:10.309595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 56/113] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:47:10 2022] GET /identity/v3/role_assignments?user.id=1e872aac4d734b22b3ef7ea653686c96&scope.project.id=9bdf56f0db064649a642e4d566966738&role.id=8eefae9314f544de97883fa2b10a2046 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:10.429558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:10.450316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6bde4a-843f-4073-9455-93b8256b9f13 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:10.501803 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751a820-768b-4142-be37-5b428fe47bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:10.502645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751a820-768b-4142-be37-5b428fe47bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:10.502942 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751a820-768b-4142-be37-5b428fe47bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:10.503225 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751a820-768b-4142-be37-5b428fe47bb6 None admin] PATH_INFO: `/v3/users/506c986a39184b05b22b043ea6c2f174` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:10.507333 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751a820-768b-4142-be37-5b428fe47bb6 None admin] RBAC: Authorizing `identity:get_user(user_id=506c986a39184b05b22b043ea6c2f174)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:10.508835 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8751a820-768b-4142-be37-5b428fe47bb6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:10.510884 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 58/114] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:10 2022] GET /identity/v3/users/506c986a39184b05b22b043ea6c2f174 => generated 275 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:10.516423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9227d8-da56-4654-816a-6786c02c5df0 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:10.669607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9227d8-da56-4654-816a-6786c02c5df0 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:10.716356 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdbd578-aac1-43db-963b-468b1ec5c3e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:10.718013 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbd578-aac1-43db-963b-468b1ec5c3e9 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:10.718500 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbd578-aac1-43db-963b-468b1ec5c3e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:10.719006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdbd578-aac1-43db-963b-468b1ec5c3e9 None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:10.720407 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdbd578-aac1-43db-963b-468b1ec5c3e9 None admin] RBAC: Authorizing `identity:get_project(project_id=9bdf56f0db064649a642e4d566966738)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:10.722044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdbd578-aac1-43db-963b-468b1ec5c3e9 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:10.724547 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 57/115] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:10 2022] GET /identity/v3/projects/9bdf56f0db064649a642e4d566966738 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:10.730585 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751a820-768b-4142-be37-5b428fe47bb6 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:10.901953 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751a820-768b-4142-be37-5b428fe47bb6 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:10.952715 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee65340-b024-4873-ab5f-dee69e4122a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:10.953630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee65340-b024-4873-ab5f-dee69e4122a1 None admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:10.953924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee65340-b024-4873-ab5f-dee69e4122a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:10.954209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee65340-b024-4873-ab5f-dee69e4122a1 None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738/users/506c986a39184b05b22b043ea6c2f174/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:10.956303 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee65340-b024-4873-ab5f-dee69e4122a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bdf56f0db064649a642e4d566966738, user_id=506c986a39184b05b22b043ea6c2f174, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:10.957770 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee65340-b024-4873-ab5f-dee69e4122a1 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:10.964402 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8ebd5-eccb-4a5d-b34c-15062df510a5 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:10.964977 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8ebd5-eccb-4a5d-b34c-15062df510a5 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:10.965332 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8ebd5-eccb-4a5d-b34c-15062df510a5 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:10.967209 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 58/116] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:47:10.971924 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50ddae-7a81-4fb2-a81b-477f6824bd2e None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:10.972208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50ddae-7a81-4fb2-a81b-477f6824bd2e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:10.972494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50ddae-7a81-4fb2-a81b-477f6824bd2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:10.978936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 59/117] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:47:10 2022] PUT /identity/v3/projects/9bdf56f0db064649a642e4d566966738/users/506c986a39184b05b22b043ea6c2f174/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:47:11.004614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-0d50ddae-7a81-4fb2-a81b-477f6824bd2e None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:11.005973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-0d50ddae-7a81-4fb2-a81b-477f6824bd2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:11.087609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:11.113349 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 59/118] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 16 22:47:11.152739 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee65340-b024-4873-ab5f-dee69e4122a1 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:11.178046 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:11.233787 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375d2e1-4e2a-4378-887a-a13cfb2be486 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:11.234091 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375d2e1-4e2a-4378-887a-a13cfb2be486 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:11.234384 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9375d2e1-4e2a-4378-887a-a13cfb2be486 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:11.235848 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 60/119] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:47:11.243367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde77078-ce49-41ff-a0d6-9aea039c3d67 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:11.243665 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde77078-ce49-41ff-a0d6-9aea039c3d67 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:11.243956 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde77078-ce49-41ff-a0d6-9aea039c3d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:11.269056 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-bde77078-ce49-41ff-a0d6-9aea039c3d67 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:11.270901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-bde77078-ce49-41ff-a0d6-9aea039c3d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:11.310218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee65340-b024-4873-ab5f-dee69e4122a1 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:11.350863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 61/120] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:47:11.369352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf57be8-1c2c-4b6f-9d00-8391e1da8613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:11.370221 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf57be8-1c2c-4b6f-9d00-8391e1da8613 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:11.370515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf57be8-1c2c-4b6f-9d00-8391e1da8613 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:11.370874 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf57be8-1c2c-4b6f-9d00-8391e1da8613 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:11.373807 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde77078-ce49-41ff-a0d6-9aea039c3d67 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:11.390142 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf57be8-1c2c-4b6f-9d00-8391e1da8613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:11.398120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-3cf57be8-1c2c-4b6f-9d00-8391e1da8613 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:11.398120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:11.398120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:11.398120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:11.398120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:11.398120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:11.398120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:11.398120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:11.398120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:11.398120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:11.398120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:11.398120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:11.398120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:11.398120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:11.398120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:11.398120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:11.398960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:11.398960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:11.398960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:11.398960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:11.398960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:11.398960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:11.398960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:11.398960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:11.398960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:11.398960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:11.398960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:11.398960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:11.398960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:11.398960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:11.398960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:11.398960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:11.399706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:11.399706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:11.399706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:11.399706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:11.399706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:11.399706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:11.399706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:11.399706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:11.399706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:11.399706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:11.399706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:11.399706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:11.399706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 60/121] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:11.406554 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf57be8-1c2c-4b6f-9d00-8391e1da8613 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:11.483716 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde77078-ce49-41ff-a0d6-9aea039c3d67 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:11.504093 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:11.533006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77315d05-6ffa-4018-9946-1f10213706df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:11.533841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77315d05-6ffa-4018-9946-1f10213706df None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:11.534133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77315d05-6ffa-4018-9946-1f10213706df None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:11.534410 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77315d05-6ffa-4018-9946-1f10213706df None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:11.545945 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77315d05-6ffa-4018-9946-1f10213706df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:11.559430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-77315d05-6ffa-4018-9946-1f10213706df None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:11.559430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:11.559430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:11.559430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:11.559430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:11.559430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:11.559430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:11.559430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:11.559430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:11.559430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:11.559430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:11.559430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:11.559430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:11.559430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:11.559430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:11.559430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:11.560397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:11.560397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:11.560397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:11.560397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:11.560397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:11.560397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:11.560397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:11.560397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:11.560397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:11.560397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:11.560397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:11.560397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:11.560397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:11.560397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:11.560397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:11.560397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:11.561164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:11.561164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:11.561164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:11.561164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:11.561164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:11.561164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:11.561164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:11.561164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:11.561164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:11.561164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:11.561164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:11.561164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:11.561164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 62/122] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:47:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:47:11.567048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b3349-b070-4d62-b07e-6e2d2745c852 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:11.567917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b3349-b070-4d62-b07e-6e2d2745c852 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:11.568168 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2b3349-b070-4d62-b07e-6e2d2745c852 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:11.574490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 63/123] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:11 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:11.575775 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a7696-6d50-4a0a-9db1-ea275d5e896a None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:11.576075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a7696-6d50-4a0a-9db1-ea275d5e896a None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:11.576361 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179a7696-6d50-4a0a-9db1-ea275d5e896a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:11.597277 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:11.600302 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-179a7696-6d50-4a0a-9db1-ea275d5e896a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:11.601513 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-179a7696-6d50-4a0a-9db1-ea275d5e896a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:11.646273 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf57be8-1c2c-4b6f-9d00-8391e1da8613 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:11.671203 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 64/124] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:11.672050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a7696-6d50-4a0a-9db1-ea275d5e896a None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:11.732677 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a0471-23ae-4dbb-a985-9ef060f79f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:11.733405 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a0471-23ae-4dbb-a985-9ef060f79f9f None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:11.733650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a0471-23ae-4dbb-a985-9ef060f79f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:11.733933 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4a0471-23ae-4dbb-a985-9ef060f79f9f None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:11.734725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a0471-23ae-4dbb-a985-9ef060f79f9f None admin] RBAC: Adding query filter params (name=member) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:11.735039 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a0471-23ae-4dbb-a985-9ef060f79f9f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:11.736418 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4a0471-23ae-4dbb-a985-9ef060f79f9f None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:11.755305 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 61/125] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:11.756145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a0471-23ae-4dbb-a985-9ef060f79f9f None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:11.826890 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179a7696-6d50-4a0a-9db1-ea275d5e896a None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:11.835140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:11.896251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14823854-84cc-430e-8e3c-0ac822135c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:11.897331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14823854-84cc-430e-8e3c-0ac822135c3b None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:11.897627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14823854-84cc-430e-8e3c-0ac822135c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:11.897913 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14823854-84cc-430e-8e3c-0ac822135c3b None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:11.898777 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14823854-84cc-430e-8e3c-0ac822135c3b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:11.899349 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14823854-84cc-430e-8e3c-0ac822135c3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:11.901040 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14823854-84cc-430e-8e3c-0ac822135c3b None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:11.927726 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 65/126] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:47:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:11.928536 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14823854-84cc-430e-8e3c-0ac822135c3b None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:11.976201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4a0471-23ae-4dbb-a985-9ef060f79f9f None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:12.047089 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45c231-828c-4a99-a2e8-8e1f64e603b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:12.047909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45c231-828c-4a99-a2e8-8e1f64e603b8 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:12.048203 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45c231-828c-4a99-a2e8-8e1f64e603b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:12.048483 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c45c231-828c-4a99-a2e8-8e1f64e603b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:12.063919 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:12.072971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c45c231-828c-4a99-a2e8-8e1f64e603b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:12.087544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-1c45c231-828c-4a99-a2e8-8e1f64e603b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:47:12.087544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:12.087544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:12.087544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:12.087544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:12.087544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:12.087544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:12.087544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:12.087544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:12.087544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:12.087544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:12.087544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:12.087544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:12.087544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:12.087544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:12.087544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:12.089674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:12.089674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:12.089674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:12.089674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:12.089674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:12.089674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:12.089674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:12.089674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:12.089674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:12.089674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:12.089674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:12.089674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:12.089674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:12.089674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:12.089674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:12.089674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:12.091580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:12.091580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:12.091580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:12.091580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:12.091580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:12.091580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:12.091580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:12.091580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:12.091580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:12.091580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:12.091580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:47:12.091580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:12.091580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 62/127] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:47:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:12.093114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45c231-828c-4a99-a2e8-8e1f64e603b8 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:12.128636 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14823854-84cc-430e-8e3c-0ac822135c3b None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:12.200909 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff65896-856c-4ff1-a799-43666579df72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:12.201675 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff65896-856c-4ff1-a799-43666579df72 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:12.201963 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff65896-856c-4ff1-a799-43666579df72 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:12.202245 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff65896-856c-4ff1-a799-43666579df72 None admin] PATH_INFO: `/v3/users/506c986a39184b05b22b043ea6c2f174` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:12.203523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff65896-856c-4ff1-a799-43666579df72 None admin] RBAC: Authorizing `identity:get_user(user_id=506c986a39184b05b22b043ea6c2f174)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:12.204967 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff65896-856c-4ff1-a799-43666579df72 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:12.206681 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 66/128] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:11 2022] GET /identity/v3/users/506c986a39184b05b22b043ea6c2f174 => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:12.207491 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff65896-856c-4ff1-a799-43666579df72 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:12.253279 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c45c231-828c-4a99-a2e8-8e1f64e603b8 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:12.283057 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:12.303283 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a9a96-54ea-444d-b10e-e71f20a75f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:12.304174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a9a96-54ea-444d-b10e-e71f20a75f4c None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:12.304467 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a9a96-54ea-444d-b10e-e71f20a75f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:12.304791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6a9a96-54ea-444d-b10e-e71f20a75f4c None admin] PATH_INFO: `/v3/users/506c986a39184b05b22b043ea6c2f174` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:12.306311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a9a96-54ea-444d-b10e-e71f20a75f4c None admin] RBAC: Authorizing `identity:get_user(user_id=506c986a39184b05b22b043ea6c2f174)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:12.308035 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6a9a96-54ea-444d-b10e-e71f20a75f4c None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:12.314242 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 63/129] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:12 2022] GET /identity/v3/users/506c986a39184b05b22b043ea6c2f174 => generated 275 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:12.315111 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a9a96-54ea-444d-b10e-e71f20a75f4c None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:12.403110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:12.433293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6a9a96-54ea-444d-b10e-e71f20a75f4c None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:12.457252 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff65896-856c-4ff1-a799-43666579df72 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:12.500917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14035504-b81d-41e7-a006-4660fb1b6be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:12.501804 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14035504-b81d-41e7-a006-4660fb1b6be1 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:12.502086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14035504-b81d-41e7-a006-4660fb1b6be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:12.502370 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14035504-b81d-41e7-a006-4660fb1b6be1 None admin] PATH_INFO: `/v3/projects/2ec770657a0e4414a0c09486f5252563` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:12.504967 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14035504-b81d-41e7-a006-4660fb1b6be1 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec770657a0e4414a0c09486f5252563)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:12.507141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14035504-b81d-41e7-a006-4660fb1b6be1 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:12.508829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 64/130] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:12 2022] GET /identity/v3/projects/2ec770657a0e4414a0c09486f5252563 => generated 312 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:12.509585 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14035504-b81d-41e7-a006-4660fb1b6be1 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:12.528529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ec621-3a5e-48d2-b03c-271ec60fd4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:12.529584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ec621-3a5e-48d2-b03c-271ec60fd4ca None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:12.530142 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ec621-3a5e-48d2-b03c-271ec60fd4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:12.530664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ec621-3a5e-48d2-b03c-271ec60fd4ca None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:12.531621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ec621-3a5e-48d2-b03c-271ec60fd4ca None admin] RBAC: Adding query filter params (name=admin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:12.532134 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ec621-3a5e-48d2-b03c-271ec60fd4ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:12.533723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ec621-3a5e-48d2-b03c-271ec60fd4ca None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:12.542912 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 67/131] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:12.544427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ec621-3a5e-48d2-b03c-271ec60fd4ca None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:12.704480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14035504-b81d-41e7-a006-4660fb1b6be1 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:12.720234 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:12.726381 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ec621-3a5e-48d2-b03c-271ec60fd4ca None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:12.745026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3077d-202e-4a08-bf6b-4fe6693ea956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:12.746412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3077d-202e-4a08-bf6b-4fe6693ea956 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:12.746904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3077d-202e-4a08-bf6b-4fe6693ea956 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:12.747323 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e3077d-202e-4a08-bf6b-4fe6693ea956 None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:12.748761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3077d-202e-4a08-bf6b-4fe6693ea956 None admin] RBAC: Authorizing `identity:get_project(project_id=9bdf56f0db064649a642e4d566966738)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:12.750467 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e3077d-202e-4a08-bf6b-4fe6693ea956 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:12.752563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 65/132] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:12 2022] GET /identity/v3/projects/9bdf56f0db064649a642e4d566966738 => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:12.753464 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3077d-202e-4a08-bf6b-4fe6693ea956 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:12.756585 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:12.763038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56163b5-dcfc-4426-9d0e-cd839ec65c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:12.764060 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56163b5-dcfc-4426-9d0e-cd839ec65c4a None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:12.764444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56163b5-dcfc-4426-9d0e-cd839ec65c4a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:12.764823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56163b5-dcfc-4426-9d0e-cd839ec65c4a None admin] PATH_INFO: `/v3/projects/2ec770657a0e4414a0c09486f5252563/users/506c986a39184b05b22b043ea6c2f174/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:12.766917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56163b5-dcfc-4426-9d0e-cd839ec65c4a None admin] RBAC: Authorizing `identity:create_grant(project_id=2ec770657a0e4414a0c09486f5252563, user_id=506c986a39184b05b22b043ea6c2f174, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:12.768477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56163b5-dcfc-4426-9d0e-cd839ec65c4a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:12.787214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 68/133] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:47:12 2022] PUT /identity/v3/projects/2ec770657a0e4414a0c09486f5252563/users/506c986a39184b05b22b043ea6c2f174/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:47:12.789420 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627b9b0-e20d-4118-be38-77853457e66b None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:12.789832 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627b9b0-e20d-4118-be38-77853457e66b None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:12.790199 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6627b9b0-e20d-4118-be38-77853457e66b None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:12.792253 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 69/134] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:12.793550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627b9b0-e20d-4118-be38-77853457e66b None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:12.927589 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6627b9b0-e20d-4118-be38-77853457e66b None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:12.938300 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e3077d-202e-4a08-bf6b-4fe6693ea956 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:12.976401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bd4439-524e-48f4-900f-ef2631e2a908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:12.977515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd4439-524e-48f4-900f-ef2631e2a908 None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:12.978440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd4439-524e-48f4-900f-ef2631e2a908 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:12.978878 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bd4439-524e-48f4-900f-ef2631e2a908 None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738/users/506c986a39184b05b22b043ea6c2f174/roles/bda1c48ca2534bc6b2ca4f3ff479b143` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:12.982614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd4439-524e-48f4-900f-ef2631e2a908 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bdf56f0db064649a642e4d566966738, user_id=506c986a39184b05b22b043ea6c2f174, role_id=bda1c48ca2534bc6b2ca4f3ff479b143)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:12.983687 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bd4439-524e-48f4-900f-ef2631e2a908 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:13.005499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 70/135] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:47:12 2022] PUT /identity/v3/projects/9bdf56f0db064649a642e4d566966738/users/506c986a39184b05b22b043ea6c2f174/roles/bda1c48ca2534bc6b2ca4f3ff479b143 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:47:13.008658 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c6a43-1598-410b-a662-0f0de6951e5b None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:13.010547 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c6a43-1598-410b-a662-0f0de6951e5b None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:13.010892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50c6a43-1598-410b-a662-0f0de6951e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:13.019278 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb329c-f7cc-4b29-abba-38c950bffb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:13.021524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb329c-f7cc-4b29-abba-38c950bffb5c None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:13.021853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb329c-f7cc-4b29-abba-38c950bffb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:13.022133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cb329c-f7cc-4b29-abba-38c950bffb5c None admin] PATH_INFO: `/v3/users/1288d3a7956c42f7ad18e2e126c7dd07` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:13.027461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb329c-f7cc-4b29-abba-38c950bffb5c None admin] RBAC: Authorizing `identity:get_user(user_id=1288d3a7956c42f7ad18e2e126c7dd07)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:13.032123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cb329c-f7cc-4b29-abba-38c950bffb5c None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:13.034162 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 66/136] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:12 2022] GET /identity/v3/users/1288d3a7956c42f7ad18e2e126c7dd07 => generated 247 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:13.040539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb329c-f7cc-4b29-abba-38c950bffb5c None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:13.053351 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-b50c6a43-1598-410b-a662-0f0de6951e5b None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:13.057442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-b50c6a43-1598-410b-a662-0f0de6951e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:13.077518 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:13.196941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:13.204193 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 71/137] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:13.240212 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c6a43-1598-410b-a662-0f0de6951e5b None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:13.247501 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cb329c-f7cc-4b29-abba-38c950bffb5c None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:13.304339 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513553f1-a112-400d-9db0-a6bb2496fd81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:13.304885 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513553f1-a112-400d-9db0-a6bb2496fd81 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:13.305178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513553f1-a112-400d-9db0-a6bb2496fd81 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:13.305459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513553f1-a112-400d-9db0-a6bb2496fd81 None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:13.306704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513553f1-a112-400d-9db0-a6bb2496fd81 None admin] RBAC: Authorizing `identity:get_project(project_id=9bdf56f0db064649a642e4d566966738)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:13.308159 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513553f1-a112-400d-9db0-a6bb2496fd81 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:13.315105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 67/138] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:13 2022] GET /identity/v3/projects/9bdf56f0db064649a642e4d566966738 => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:13.320438 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513553f1-a112-400d-9db0-a6bb2496fd81 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:13.382481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50c6a43-1598-410b-a662-0f0de6951e5b None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:13.425768 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048347a0-ac76-4a01-85cc-b2cf852fccc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:13.427512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048347a0-ac76-4a01-85cc-b2cf852fccc9 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:13.428026 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048347a0-ac76-4a01-85cc-b2cf852fccc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:13.428530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048347a0-ac76-4a01-85cc-b2cf852fccc9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:13.430777 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048347a0-ac76-4a01-85cc-b2cf852fccc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:13.433000 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048347a0-ac76-4a01-85cc-b2cf852fccc9 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:13.435227 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 72/139] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:47:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:13.436897 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca326cd-73d2-4241-8ea0-2802af2f8440 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:13.439570 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca326cd-73d2-4241-8ea0-2802af2f8440 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:13.440085 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca326cd-73d2-4241-8ea0-2802af2f8440 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:13.444491 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 73/140] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:13 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:13.446468 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca326cd-73d2-4241-8ea0-2802af2f8440 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:13.467266 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:13.550253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513553f1-a112-400d-9db0-a6bb2496fd81 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:13.597304 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca326cd-73d2-4241-8ea0-2802af2f8440 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:13.603733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdc6df3-afab-44fd-91b8-5672a9fc199b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:13.604638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc6df3-afab-44fd-91b8-5672a9fc199b None admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:13.604925 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc6df3-afab-44fd-91b8-5672a9fc199b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:13.605203 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdc6df3-afab-44fd-91b8-5672a9fc199b None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738/users/1288d3a7956c42f7ad18e2e126c7dd07/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:13.607618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc6df3-afab-44fd-91b8-5672a9fc199b None admin] RBAC: Authorizing `identity:create_grant(project_id=9bdf56f0db064649a642e4d566966738, user_id=1288d3a7956c42f7ad18e2e126c7dd07, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:13.608795 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdc6df3-afab-44fd-91b8-5672a9fc199b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:13.618683 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:13.626136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 68/141] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:47:13 2022] PUT /identity/v3/projects/9bdf56f0db064649a642e4d566966738/users/1288d3a7956c42f7ad18e2e126c7dd07/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 22:47:13.633035 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6257f-5dc7-4e0c-9074-434326266752 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:13.633376 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6257f-5dc7-4e0c-9074-434326266752 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:13.633662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b6257f-5dc7-4e0c-9074-434326266752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:13.638133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1c8e76-ecb6-4c3e-90b0-7fd2c21172c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:13.639326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c8e76-ecb6-4c3e-90b0-7fd2c21172c8 None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:13.639920 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c8e76-ecb6-4c3e-90b0-7fd2c21172c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:13.640413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1c8e76-ecb6-4c3e-90b0-7fd2c21172c8 None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:13.641462 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c8e76-ecb6-4c3e-90b0-7fd2c21172c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:13.643180 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1c8e76-ecb6-4c3e-90b0-7fd2c21172c8 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:13.664516 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-00b6257f-5dc7-4e0c-9074-434326266752 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:13.673203 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 74/142] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 22:47:13 2022] POST /identity/v3/users => generated 283 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:47:13.674959 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2f130-2ebc-4834-8478-8d7b53c9f881 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:13.675286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-00b6257f-5dc7-4e0c-9074-434326266752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:13.677057 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2f130-2ebc-4834-8478-8d7b53c9f881 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:13.679004 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c2f130-2ebc-4834-8478-8d7b53c9f881 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:13.680855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 75/143] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:13.692183 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c9dff-ec38-4fb5-afab-ec2dbb6ab4ad None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:13.693382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c9dff-ec38-4fb5-afab-ec2dbb6ab4ad None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:13.693875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1c9dff-ec38-4fb5-afab-ec2dbb6ab4ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:13.735847 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-5e1c9dff-ec38-4fb5-afab-ec2dbb6ab4ad None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:13.737093 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-5e1c9dff-ec38-4fb5-afab-ec2dbb6ab4ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:13.817449 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 69/144] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:13.860205 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6257f-5dc7-4e0c-9074-434326266752 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:13.921739 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:13.936385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 76/145] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:13.983130 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c9dff-ec38-4fb5-afab-ec2dbb6ab4ad None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:14.018650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b6257f-5dc7-4e0c-9074-434326266752 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:14.052559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:14.062789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54853f9e-f7d5-4a98-9972-0412eb705266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:14.063613 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54853f9e-f7d5-4a98-9972-0412eb705266 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:14.063901 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54853f9e-f7d5-4a98-9972-0412eb705266 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:14.064537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54853f9e-f7d5-4a98-9972-0412eb705266 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:14.078923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54853f9e-f7d5-4a98-9972-0412eb705266 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:14.088757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-54853f9e-f7d5-4a98-9972-0412eb705266 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 22:47:14.088757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:14.088757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:14.088757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:14.088757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:14.088757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:14.088757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:14.088757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:14.088757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:14.088757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:14.088757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:14.088757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:14.088757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:14.088757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:14.088757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:14.088757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:14.089845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:14.089845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:14.089845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:14.089845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:14.089845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:14.089845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:14.089845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:14.089845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:14.089845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:14.089845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:14.089845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:14.089845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:14.089845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:14.089845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:14.089845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:14.089845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:14.090789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:14.090789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:14.090789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:14.090789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:14.090789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:14.090789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:14.090789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:14.090789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:14.090789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:14.090789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:14.090789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 22:47:14.090789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:14.093171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 70/146] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:47:14.099067 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54853f9e-f7d5-4a98-9972-0412eb705266 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:14.130066 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1c9dff-ec38-4fb5-afab-ec2dbb6ab4ad None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:14.171016 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd93a68-9393-41c7-b2f3-40bb216d01b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:14.171826 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd93a68-9393-41c7-b2f3-40bb216d01b1 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:14.172116 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd93a68-9393-41c7-b2f3-40bb216d01b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:14.172395 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd93a68-9393-41c7-b2f3-40bb216d01b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:14.195287 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd93a68-9393-41c7-b2f3-40bb216d01b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:14.209258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-dfd93a68-9393-41c7-b2f3-40bb216d01b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:14.209258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:14.209258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:14.209258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:14.209258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:14.209258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:14.209258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:14.209258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:14.209258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:14.209258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:14.209258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:14.209258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:14.209258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:14.209258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:14.209258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:14.209258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:14.210137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:14.210137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:14.210137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:14.210137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:14.210137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:14.210137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:14.210137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:14.210137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:14.210137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:14.210137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:14.210137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:14.210137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:14.210137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:14.210137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:14.210137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:14.210137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:14.210930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:14.210930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:14.210930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:14.210930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:14.210930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:14.210930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:14.210930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:14.210930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:14.210930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:14.210930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:14.210930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:14.210930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:14.210930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 77/147] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:47:14.217754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd93a68-9393-41c7-b2f3-40bb216d01b1 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:14.295216 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54853f9e-f7d5-4a98-9972-0412eb705266 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:14.339137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812788f3-4769-4137-853e-004fce8bd789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:14.339803 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812788f3-4769-4137-853e-004fce8bd789 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:14.340099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812788f3-4769-4137-853e-004fce8bd789 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:14.340433 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812788f3-4769-4137-853e-004fce8bd789 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:14.341194 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812788f3-4769-4137-853e-004fce8bd789 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:14.341565 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812788f3-4769-4137-853e-004fce8bd789 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:14.343148 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812788f3-4769-4137-853e-004fce8bd789 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:14.356427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 71/148] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:14.363012 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812788f3-4769-4137-853e-004fce8bd789 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:14.372312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:14.428165 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:14.480539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd93a68-9393-41c7-b2f3-40bb216d01b1 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:14.494677 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812788f3-4769-4137-853e-004fce8bd789 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:14.519984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579be8f3-546c-44d2-aed7-4c66ff577eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:14.524463 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579be8f3-546c-44d2-aed7-4c66ff577eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:14.524820 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579be8f3-546c-44d2-aed7-4c66ff577eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:14.531446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579be8f3-546c-44d2-aed7-4c66ff577eb0 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:14.532598 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579be8f3-546c-44d2-aed7-4c66ff577eb0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:14.532927 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579be8f3-546c-44d2-aed7-4c66ff577eb0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:14.534970 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579be8f3-546c-44d2-aed7-4c66ff577eb0 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:14.540913 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf246e-d5a4-4eb0-9867-5369fd63f3d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:14.541840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf246e-d5a4-4eb0-9867-5369fd63f3d0 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:14.542129 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf246e-d5a4-4eb0-9867-5369fd63f3d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:14.542411 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf246e-d5a4-4eb0-9867-5369fd63f3d0 None admin] PATH_INFO: `/v3/users/1e872aac4d734b22b3ef7ea653686c96` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:14.543799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf246e-d5a4-4eb0-9867-5369fd63f3d0 None admin] RBAC: Authorizing `identity:get_user(user_id=1e872aac4d734b22b3ef7ea653686c96)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:14.545294 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf246e-d5a4-4eb0-9867-5369fd63f3d0 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:14.548214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 72/149] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:14 2022] GET /identity/v3/users/1e872aac4d734b22b3ef7ea653686c96 => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:14.550189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 78/150] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:14.553776 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf246e-d5a4-4eb0-9867-5369fd63f3d0 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:14.555902 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579be8f3-546c-44d2-aed7-4c66ff577eb0 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:14.710104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579be8f3-546c-44d2-aed7-4c66ff577eb0 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:14.734260 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf246e-d5a4-4eb0-9867-5369fd63f3d0 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:14.736469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:14.749444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18a6bd-dad2-4740-b1b0-bda41bec47b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:14.750397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18a6bd-dad2-4740-b1b0-bda41bec47b3 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:14.750729 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18a6bd-dad2-4740-b1b0-bda41bec47b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:14.751010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e18a6bd-dad2-4740-b1b0-bda41bec47b3 None admin] PATH_INFO: `/v3/users/506c986a39184b05b22b043ea6c2f174` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:14.752462 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18a6bd-dad2-4740-b1b0-bda41bec47b3 None admin] RBAC: Authorizing `identity:get_user(user_id=506c986a39184b05b22b043ea6c2f174)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:14.753927 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e18a6bd-dad2-4740-b1b0-bda41bec47b3 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:14.755639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:14.756856 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 79/151] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:14 2022] GET /identity/v3/users/506c986a39184b05b22b043ea6c2f174 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:14.762454 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18a6bd-dad2-4740-b1b0-bda41bec47b3 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:14.783567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb3c70-aa71-42c2-ad2a-658ace740b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:14.784690 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb3c70-aa71-42c2-ad2a-658ace740b40 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:14.784987 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb3c70-aa71-42c2-ad2a-658ace740b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:14.785425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffb3c70-aa71-42c2-ad2a-658ace740b40 None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:14.787339 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb3c70-aa71-42c2-ad2a-658ace740b40 None admin] RBAC: Authorizing `identity:get_project(project_id=9bdf56f0db064649a642e4d566966738)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:14.789259 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffb3c70-aa71-42c2-ad2a-658ace740b40 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:14.792239 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 73/152] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:14 2022] GET /identity/v3/projects/9bdf56f0db064649a642e4d566966738 => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:14.799295 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb3c70-aa71-42c2-ad2a-658ace740b40 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:14.958007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e18a6bd-dad2-4740-b1b0-bda41bec47b3 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:15.024742 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffb3c70-aa71-42c2-ad2a-658ace740b40 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:15.043955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27be1f-ebe6-432a-b112-4d9a3bf169cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:15.044624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27be1f-ebe6-432a-b112-4d9a3bf169cf None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:15.044908 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27be1f-ebe6-432a-b112-4d9a3bf169cf None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:15.045186 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b27be1f-ebe6-432a-b112-4d9a3bf169cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:15.103906 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513a918-bbc5-4e71-ad13-fc58823a987e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:15.105019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513a918-bbc5-4e71-ad13-fc58823a987e None admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:15.105304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513a918-bbc5-4e71-ad13-fc58823a987e None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:15.105588 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5513a918-bbc5-4e71-ad13-fc58823a987e None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738/users/1e872aac4d734b22b3ef7ea653686c96/roles/8eefae9314f544de97883fa2b10a2046` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:15.109341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5513a918-bbc5-4e71-ad13-fc58823a987e None admin] RBAC: Authorizing `identity:create_grant(project_id=9bdf56f0db064649a642e4d566966738, user_id=1e872aac4d734b22b3ef7ea653686c96, role_id=8eefae9314f544de97883fa2b10a2046)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:15.116034 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5513a918-bbc5-4e71-ad13-fc58823a987e None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:15.143210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 74/153] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:47:14 2022] PUT /identity/v3/projects/9bdf56f0db064649a642e4d566966738/users/1e872aac4d734b22b3ef7ea653686c96/roles/8eefae9314f544de97883fa2b10a2046 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 22:47:15.166287 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:15.178385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27be1f-ebe6-432a-b112-4d9a3bf169cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:15.180001 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b27be1f-ebe6-432a-b112-4d9a3bf169cf None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:15.319836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 80/154] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 22:47:15.325596 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513a918-bbc5-4e71-ad13-fc58823a987e None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:15.380788 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:15.445254 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5513a918-bbc5-4e71-ad13-fc58823a987e None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:15.494262 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87b335-7579-41f2-8807-b924870d6739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:15.495177 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87b335-7579-41f2-8807-b924870d6739 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:15.495461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87b335-7579-41f2-8807-b924870d6739 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:15.496034 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87b335-7579-41f2-8807-b924870d6739 None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:15.497195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e87b335-7579-41f2-8807-b924870d6739 None admin] RBAC: Authorizing `identity:get_project(project_id=9bdf56f0db064649a642e4d566966738)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:15.498681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e87b335-7579-41f2-8807-b924870d6739 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:15.500443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 75/155] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:15 2022] GET /identity/v3/projects/9bdf56f0db064649a642e4d566966738 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:15.506875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27be1f-ebe6-432a-b112-4d9a3bf169cf None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:15.604382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:15.613693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b27be1f-ebe6-432a-b112-4d9a3bf169cf None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:15.656833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496140b-c1ed-442d-a665-963c9776ca12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:15.657493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496140b-c1ed-442d-a665-963c9776ca12 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:15.657774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496140b-c1ed-442d-a665-963c9776ca12 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:15.658050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496140b-c1ed-442d-a665-963c9776ca12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:15.658910 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496140b-c1ed-442d-a665-963c9776ca12 None admin] RBAC: Adding query filter params (role.id=bff4f694fb57451e9a0783dc014187f2, scope.project.id=9bdf56f0db064649a642e4d566966738, user.id=506c986a39184b05b22b043ea6c2f174) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:15.659224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496140b-c1ed-442d-a665-963c9776ca12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:15.660671 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496140b-c1ed-442d-a665-963c9776ca12 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:15.671308 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 81/156] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:47:15 2022] GET /identity/v3/role_assignments?user.id=506c986a39184b05b22b043ea6c2f174&scope.project.id=9bdf56f0db064649a642e4d566966738&role.id=bff4f694fb57451e9a0783dc014187f2 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:16.284668 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54db20-d540-42a1-a860-10a5fa4547ca None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:16.285615 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54db20-d540-42a1-a860-10a5fa4547ca None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:16.286085 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54db20-d540-42a1-a860-10a5fa4547ca None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:16.288061 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 76/157] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:16.293098 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8392fb-11e5-41a5-b8ab-4e94bfd2a4fb None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:16.293999 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8392fb-11e5-41a5-b8ab-4e94bfd2a4fb None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:16.294480 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8392fb-11e5-41a5-b8ab-4e94bfd2a4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:16.319577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-3f8392fb-11e5-41a5-b8ab-4e94bfd2a4fb None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:16.321125 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-3f8392fb-11e5-41a5-b8ab-4e94bfd2a4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:16.404856 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 82/158] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:16.453758 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54db20-d540-42a1-a860-10a5fa4547ca None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:16.575870 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54db20-d540-42a1-a860-10a5fa4547ca None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:16.592131 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:16.619783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621a65a-1f24-486e-9d24-6c9d700bca41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:16.620945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621a65a-1f24-486e-9d24-6c9d700bca41 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:16.621421 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621a65a-1f24-486e-9d24-6c9d700bca41 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:16.621899 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621a65a-1f24-486e-9d24-6c9d700bca41 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:16.637257 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621a65a-1f24-486e-9d24-6c9d700bca41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:16.645485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-a621a65a-1f24-486e-9d24-6c9d700bca41 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:16.645485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:16.645485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:16.645485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:16.645485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:16.645485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:16.645485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:16.645485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:16.645485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:16.645485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:16.645485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:16.645485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:16.645485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:16.645485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:16.645485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:16.645485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:16.646509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:16.646509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:16.646509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:16.646509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:16.646509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:16.646509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:16.646509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:16.646509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:16.646509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:16.646509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:16.646509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:16.646509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:16.646509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:16.646509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:16.646509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:16.646509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:16.647296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:16.647296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:16.647296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:16.647296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:16.647296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:16.647296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:16.647296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:16.647296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:16.647296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:16.647296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:16.647296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:16.647296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:16.649847 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 77/159] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:47:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:16.655460 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8392fb-11e5-41a5-b8ab-4e94bfd2a4fb None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:16.672951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be0972-434f-433e-a2e2-d7e10ef59dd5 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:16.673537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be0972-434f-433e-a2e2-d7e10ef59dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:16.674028 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71be0972-434f-433e-a2e2-d7e10ef59dd5 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:16.675620 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 78/160] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:47:16.680689 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fc3de-c59a-4818-ba5d-5dda699f0f55 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:16.681223 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fc3de-c59a-4818-ba5d-5dda699f0f55 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:16.681720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fc3de-c59a-4818-ba5d-5dda699f0f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:16.716008 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-c37fc3de-c59a-4818-ba5d-5dda699f0f55 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:16.718237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c37fc3de-c59a-4818-ba5d-5dda699f0f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:16.830179 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:16.853229 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 79/161] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:47:16.872745 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8392fb-11e5-41a5-b8ab-4e94bfd2a4fb None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:16.888417 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37fc3de-c59a-4818-ba5d-5dda699f0f55 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:16.924626 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4296c5f-08fe-4a56-b3cf-a9da5e50325e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:16.925720 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4296c5f-08fe-4a56-b3cf-a9da5e50325e None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:16.926268 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4296c5f-08fe-4a56-b3cf-a9da5e50325e None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:16.926784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4296c5f-08fe-4a56-b3cf-a9da5e50325e None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:16.928282 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4296c5f-08fe-4a56-b3cf-a9da5e50325e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:16.928802 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4296c5f-08fe-4a56-b3cf-a9da5e50325e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:16.930402 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4296c5f-08fe-4a56-b3cf-a9da5e50325e None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:16.946514 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 83/162] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:47:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:16.959922 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4296c5f-08fe-4a56-b3cf-a9da5e50325e None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:17.012589 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37fc3de-c59a-4818-ba5d-5dda699f0f55 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:17.014955 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:17.070712 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638dd93-4c7c-4ff8-a74d-af19cb547319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:17.071524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638dd93-4c7c-4ff8-a74d-af19cb547319 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:17.071803 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638dd93-4c7c-4ff8-a74d-af19cb547319 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:17.072075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2638dd93-4c7c-4ff8-a74d-af19cb547319 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:17.086664 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2638dd93-4c7c-4ff8-a74d-af19cb547319 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:17.096010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-2638dd93-4c7c-4ff8-a74d-af19cb547319 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:17.096010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:17.096010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:17.096010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:17.096010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:17.096010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:17.096010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:17.096010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:17.096010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:17.096010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:17.096010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:17.096010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:17.096010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:17.096010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:17.096010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:17.096010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:17.096932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:17.096932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:17.096932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:17.096932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:17.096932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:17.096932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:17.096932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:17.096932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:17.096932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:17.096932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:17.096932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:17.096932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:17.096932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:17.096932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:17.096932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:17.096932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:17.097685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:17.097685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:17.097685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:17.097685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:17.097685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:17.097685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:17.097685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:17.097685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:17.097685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:17.097685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:17.097685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:17.097685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:17.097685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 80/163] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:47:17.103126 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638dd93-4c7c-4ff8-a74d-af19cb547319 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:17.150281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4296c5f-08fe-4a56-b3cf-a9da5e50325e None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:17.204125 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2638dd93-4c7c-4ff8-a74d-af19cb547319 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:17.222661 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47551099-f51a-40dc-b01d-fc312fb0fab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:17.223565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47551099-f51a-40dc-b01d-fc312fb0fab1 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:17.223851 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47551099-f51a-40dc-b01d-fc312fb0fab1 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:17.224182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47551099-f51a-40dc-b01d-fc312fb0fab1 None admin] PATH_INFO: `/v3/users/506c986a39184b05b22b043ea6c2f174` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:17.225409 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47551099-f51a-40dc-b01d-fc312fb0fab1 None admin] RBAC: Authorizing `identity:get_user(user_id=506c986a39184b05b22b043ea6c2f174)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:17.226898 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47551099-f51a-40dc-b01d-fc312fb0fab1 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:17.233250 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 84/164] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:16 2022] GET /identity/v3/users/506c986a39184b05b22b043ea6c2f174 => generated 275 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:17.241610 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47551099-f51a-40dc-b01d-fc312fb0fab1 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:17.248233 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba0ba0-e605-4bd4-a11c-77f69449ef61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:17.248919 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba0ba0-e605-4bd4-a11c-77f69449ef61 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:17.249201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba0ba0-e605-4bd4-a11c-77f69449ef61 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:17.249474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ba0ba0-e605-4bd4-a11c-77f69449ef61 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:17.250234 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba0ba0-e605-4bd4-a11c-77f69449ef61 None admin] RBAC: Adding query filter params (name=member) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:17.250543 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba0ba0-e605-4bd4-a11c-77f69449ef61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:17.253822 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ba0ba0-e605-4bd4-a11c-77f69449ef61 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:17.271537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 81/165] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:17.279513 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:17.279944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba0ba0-e605-4bd4-a11c-77f69449ef61 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:17.338697 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:17.460687 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47551099-f51a-40dc-b01d-fc312fb0fab1 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:17.523332 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ba0ba0-e605-4bd4-a11c-77f69449ef61 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:17.525497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c679fe5-6233-411e-90d4-9d18006b59db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:17.526700 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c679fe5-6233-411e-90d4-9d18006b59db None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:17.527129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c679fe5-6233-411e-90d4-9d18006b59db None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:17.527546 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c679fe5-6233-411e-90d4-9d18006b59db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:17.611641 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c2914-b00f-4164-a489-9f04469c06dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:17.612533 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c2914-b00f-4164-a489-9f04469c06dd None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:17.612824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c2914-b00f-4164-a489-9f04469c06dd None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:17.613105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94c2914-b00f-4164-a489-9f04469c06dd None admin] PATH_INFO: `/v3/users/506c986a39184b05b22b043ea6c2f174` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:17.615159 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c2914-b00f-4164-a489-9f04469c06dd None admin] RBAC: Authorizing `identity:get_user(user_id=506c986a39184b05b22b043ea6c2f174)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:17.616620 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94c2914-b00f-4164-a489-9f04469c06dd None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:17.618295 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 82/166] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:17 2022] GET /identity/v3/users/506c986a39184b05b22b043ea6c2f174 => generated 275 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:17.629290 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c2914-b00f-4164-a489-9f04469c06dd None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:17.691171 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c679fe5-6233-411e-90d4-9d18006b59db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:17.694760 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c679fe5-6233-411e-90d4-9d18006b59db None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:17.722970 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:17.821640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 85/167] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 581 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 22:47:17.823040 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad63046-f872-4772-b134-66f2bfe451a5 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:17.823331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad63046-f872-4772-b134-66f2bfe451a5 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:17.823613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad63046-f872-4772-b134-66f2bfe451a5 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:17.825143 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 86/168] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:17.825568 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:17.832058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc91a0d-25f8-4145-936c-83671b51d93d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:17.832345 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc91a0d-25f8-4145-936c-83671b51d93d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:17.832635 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc91a0d-25f8-4145-936c-83671b51d93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:17.844761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94c2914-b00f-4164-a489-9f04469c06dd None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:17.859637 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-8bc91a0d-25f8-4145-936c-83671b51d93d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:17.860861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-8bc91a0d-25f8-4145-936c-83671b51d93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:17.937395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f76a9d-baa7-48f0-aced-03af1a52a634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:17.938397 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f76a9d-baa7-48f0-aced-03af1a52a634 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:17.938907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f76a9d-baa7-48f0-aced-03af1a52a634 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:17.939382 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f76a9d-baa7-48f0-aced-03af1a52a634 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:17.965115 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 87/169] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:47:17.965935 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc91a0d-25f8-4145-936c-83671b51d93d None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:18.039291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:18.103696 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f76a9d-baa7-48f0-aced-03af1a52a634 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:18.105181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f76a9d-baa7-48f0-aced-03af1a52a634 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:18.113859 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc91a0d-25f8-4145-936c-83671b51d93d None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:18.192337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2de9e6-f773-4a82-8605-d07414d850e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:18.193673 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2de9e6-f773-4a82-8605-d07414d850e5 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:18.193954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2de9e6-f773-4a82-8605-d07414d850e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:18.194230 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2de9e6-f773-4a82-8605-d07414d850e5 None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:18.199322 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2de9e6-f773-4a82-8605-d07414d850e5 None admin] RBAC: Authorizing `identity:get_project(project_id=9bdf56f0db064649a642e4d566966738)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:18.200810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2de9e6-f773-4a82-8605-d07414d850e5 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:18.202714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 88/170] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:17 2022] GET /identity/v3/projects/9bdf56f0db064649a642e4d566966738 => generated 298 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:18.203404 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2de9e6-f773-4a82-8605-d07414d850e5 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:18.362535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:18.379875 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 83/171] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 751 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:18.381138 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f76a9d-baa7-48f0-aced-03af1a52a634 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:18.454971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2de9e6-f773-4a82-8605-d07414d850e5 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:18.473836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:18.496955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac7612-3a4a-4e7f-a7aa-b35d14395d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:18.498922 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac7612-3a4a-4e7f-a7aa-b35d14395d24 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:18.499456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac7612-3a4a-4e7f-a7aa-b35d14395d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:18.499935 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aac7612-3a4a-4e7f-a7aa-b35d14395d24 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:18.514541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aac7612-3a4a-4e7f-a7aa-b35d14395d24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:18.525401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-7aac7612-3a4a-4e7f-a7aa-b35d14395d24 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:47:18.525401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:18.525401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:18.525401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:18.525401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:18.525401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:18.525401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:18.525401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:18.525401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:18.525401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:18.525401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:18.525401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:18.525401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:18.525401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:18.525401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:18.525401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:18.526550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:18.526550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:18.526550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:18.526550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:18.526550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:18.526550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:18.526550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:18.526550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:18.526550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:18.526550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:18.526550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:18.526550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:18.526550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:18.526550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:18.526550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:18.526550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:18.527952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:18.527952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:18.527952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:18.527952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:18.527952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:18.527952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:18.527952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:18.527952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:18.527952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:18.527952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:18.527952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:47:18.527952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:18.527952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 89/172] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:47:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:18.528840 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac7612-3a4a-4e7f-a7aa-b35d14395d24 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:18.536894 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f76a9d-baa7-48f0-aced-03af1a52a634 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:18.572543 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b58b1b-fd7a-4cd9-a7a3-c102351c3ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:18.573235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b58b1b-fd7a-4cd9-a7a3-c102351c3ffe None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:18.573522 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b58b1b-fd7a-4cd9-a7a3-c102351c3ffe None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:18.573802 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b58b1b-fd7a-4cd9-a7a3-c102351c3ffe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:18.574658 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b58b1b-fd7a-4cd9-a7a3-c102351c3ffe None admin] RBAC: Adding query filter params (role.id=bda1c48ca2534bc6b2ca4f3ff479b143, scope.project.id=9bdf56f0db064649a642e4d566966738, user.id=506c986a39184b05b22b043ea6c2f174) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:18.574973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b58b1b-fd7a-4cd9-a7a3-c102351c3ffe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:18.576426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b58b1b-fd7a-4cd9-a7a3-c102351c3ffe None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:18.582357 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 84/173] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:47:18 2022] GET /identity/v3/role_assignments?user.id=506c986a39184b05b22b043ea6c2f174&scope.project.id=9bdf56f0db064649a642e4d566966738&role.id=bda1c48ca2534bc6b2ca4f3ff479b143 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:18.582357 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5ebb6-d52b-4d40-8327-960ef4bde031 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:18.582357 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5ebb6-d52b-4d40-8327-960ef4bde031 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:18.582357 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5ebb6-d52b-4d40-8327-960ef4bde031 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:18.585558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 85/174] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:18.590015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851cd21-6c5c-4a0b-a64d-cb4b0d7efd28 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:18.590015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851cd21-6c5c-4a0b-a64d-cb4b0d7efd28 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:18.590015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851cd21-6c5c-4a0b-a64d-cb4b0d7efd28 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:18.590015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 86/175] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:18.590015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49d662-c542-4d10-9d07-b16936510bfc None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:18.590542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49d662-c542-4d10-9d07-b16936510bfc None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:18.590751 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a49d662-c542-4d10-9d07-b16936510bfc None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:18.596202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 87/176] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:18.597202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49d662-c542-4d10-9d07-b16936510bfc None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:18.655142 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aac7612-3a4a-4e7f-a7aa-b35d14395d24 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:18.674726 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:18.701996 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ff00df-b942-4ddd-9a60-f6576da436e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:18.702945 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff00df-b942-4ddd-9a60-f6576da436e3 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:18.703259 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff00df-b942-4ddd-9a60-f6576da436e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:18.703982 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ff00df-b942-4ddd-9a60-f6576da436e3 None admin] PATH_INFO: `/v3/projects/2ec770657a0e4414a0c09486f5252563` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:18.704818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff00df-b942-4ddd-9a60-f6576da436e3 None admin] RBAC: Authorizing `identity:get_project(project_id=2ec770657a0e4414a0c09486f5252563)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:18.706393 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ff00df-b942-4ddd-9a60-f6576da436e3 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:18.708182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 90/177] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:18 2022] GET /identity/v3/projects/2ec770657a0e4414a0c09486f5252563 => generated 312 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:18.709549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb26b7-3a6b-4fcd-80ca-865dd2cfb024 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:18.709848 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb26b7-3a6b-4fcd-80ca-865dd2cfb024 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:18.710132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb26b7-3a6b-4fcd-80ca-865dd2cfb024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:18.727925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-59eb26b7-3a6b-4fcd-80ca-865dd2cfb024 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:18.729164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-59eb26b7-3a6b-4fcd-80ca-865dd2cfb024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:18.739758 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a49d662-c542-4d10-9d07-b16936510bfc None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:18.773043 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:18.794786 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1b0884-22e0-4360-aafd-0a2e75cff54e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:18.795522 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b0884-22e0-4360-aafd-0a2e75cff54e None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:18.795809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b0884-22e0-4360-aafd-0a2e75cff54e None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:18.796139 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1b0884-22e0-4360-aafd-0a2e75cff54e None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:18.796955 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b0884-22e0-4360-aafd-0a2e75cff54e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:18.797540 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b0884-22e0-4360-aafd-0a2e75cff54e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:18.798752 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1b0884-22e0-4360-aafd-0a2e75cff54e None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:18.809026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 88/178] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:18.810234 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00952114-f0e1-46ed-987b-4cf5b5eee67b None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:18.810475 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00952114-f0e1-46ed-987b-4cf5b5eee67b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:18.810793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00952114-f0e1-46ed-987b-4cf5b5eee67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:18.811560 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 91/179] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:18.812831 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b815b-535f-433e-a4a8-d8e4be8265fa None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:18.813125 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b815b-535f-433e-a4a8-d8e4be8265fa None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:18.813407 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b815b-535f-433e-a4a8-d8e4be8265fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:18.829890 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-00952114-f0e1-46ed-987b-4cf5b5eee67b None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:18.831257 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-00952114-f0e1-46ed-987b-4cf5b5eee67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:18.845133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-f51b815b-535f-433e-a4a8-d8e4be8265fa None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:18.846227 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f51b815b-535f-433e-a4a8-d8e4be8265fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:18.916856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 89/180] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:18.926788 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00952114-f0e1-46ed-987b-4cf5b5eee67b None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:18.929939 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 92/181] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:18.930890 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b815b-535f-433e-a4a8-d8e4be8265fa None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:18.986261 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:19.040421 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00952114-f0e1-46ed-987b-4cf5b5eee67b None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:19.040611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:19.043856 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b815b-535f-433e-a4a8-d8e4be8265fa None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:19.075581 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32e055-e49a-4ab7-9528-f9821b26528c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:19.076279 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32e055-e49a-4ab7-9528-f9821b26528c None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:19.076554 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32e055-e49a-4ab7-9528-f9821b26528c None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:19.076834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a32e055-e49a-4ab7-9528-f9821b26528c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:19.077669 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32e055-e49a-4ab7-9528-f9821b26528c None admin] RBAC: Adding query filter params (role.id=bff4f694fb57451e9a0783dc014187f2, scope.project.id=2ec770657a0e4414a0c09486f5252563, user.id=506c986a39184b05b22b043ea6c2f174) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:19.077987 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32e055-e49a-4ab7-9528-f9821b26528c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:19.079452 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a32e055-e49a-4ab7-9528-f9821b26528c None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:19.080823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df945d5d-2f19-4725-aa03-2f3415a20595 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:19.081701 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df945d5d-2f19-4725-aa03-2f3415a20595 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:19.081990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df945d5d-2f19-4725-aa03-2f3415a20595 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:19.082280 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df945d5d-2f19-4725-aa03-2f3415a20595 None admin] PATH_INFO: `/v3/users/1288d3a7956c42f7ad18e2e126c7dd07` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:19.083504 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df945d5d-2f19-4725-aa03-2f3415a20595 None admin] RBAC: Authorizing `identity:get_user(user_id=1288d3a7956c42f7ad18e2e126c7dd07)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:19.084998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df945d5d-2f19-4725-aa03-2f3415a20595 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:19.090647 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 93/182] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:18 2022] GET /identity/v3/users/1288d3a7956c42f7ad18e2e126c7dd07 => generated 247 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:19.090647 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df945d5d-2f19-4725-aa03-2f3415a20595 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:19.091311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 90/183] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:47:18 2022] GET /identity/v3/role_assignments?user.id=506c986a39184b05b22b043ea6c2f174&scope.project.id=2ec770657a0e4414a0c09486f5252563&role.id=bff4f694fb57451e9a0783dc014187f2 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:19.100174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32e055-e49a-4ab7-9528-f9821b26528c None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:19.222292 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df945d5d-2f19-4725-aa03-2f3415a20595 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:19.232342 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a32e055-e49a-4ab7-9528-f9821b26528c None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:19.249076 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:19.262364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcdb86-9c37-4d95-97fb-dd3162be6234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:19.263199 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcdb86-9c37-4d95-97fb-dd3162be6234 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:19.263497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcdb86-9c37-4d95-97fb-dd3162be6234 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:19.263775 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dcdb86-9c37-4d95-97fb-dd3162be6234 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:19.264994 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dcdb86-9c37-4d95-97fb-dd3162be6234 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:19.266445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dcdb86-9c37-4d95-97fb-dd3162be6234 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:19.268297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 94/184] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:47:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:19.268974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcdb86-9c37-4d95-97fb-dd3162be6234 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:19.279202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69835161-4709-4e1b-82c7-18544936f0dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:19.280377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69835161-4709-4e1b-82c7-18544936f0dc None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:19.280377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69835161-4709-4e1b-82c7-18544936f0dc None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:19.280639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69835161-4709-4e1b-82c7-18544936f0dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:19.292054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69835161-4709-4e1b-82c7-18544936f0dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:19.300098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-69835161-4709-4e1b-82c7-18544936f0dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:19.300098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:19.300098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:19.300098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:19.300098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:19.300098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:19.300098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:19.300098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:19.300098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:19.300098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:19.300098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:19.300098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:19.300098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:19.300098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:19.300098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:19.300098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:19.301123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:19.301123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:19.301123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:19.301123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:19.301123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:19.301123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:19.301123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:19.301123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:19.301123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:19.301123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:19.301123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:19.301123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:19.301123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:19.301123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:19.301123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:19.301123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:19.302024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:19.302024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:19.302024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:19.302024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:19.302024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:19.302024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:19.302024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:19.302024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:19.302024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:19.302024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:19.302024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:19.302024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:19.302024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 91/185] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:47:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:19.303389 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69835161-4709-4e1b-82c7-18544936f0dc None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:19.309267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:19.375857 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dcdb86-9c37-4d95-97fb-dd3162be6234 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:19.409074 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a9c049-24e1-4daf-acdc-743ad136edad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:19.409867 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9c049-24e1-4daf-acdc-743ad136edad None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:19.410160 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9c049-24e1-4daf-acdc-743ad136edad None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:19.410446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a9c049-24e1-4daf-acdc-743ad136edad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:19.413016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69835161-4709-4e1b-82c7-18544936f0dc None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:19.420543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a9c049-24e1-4daf-acdc-743ad136edad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:19.428597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-73a9c049-24e1-4daf-acdc-743ad136edad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:47:19.428597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:19.428597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:19.428597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:19.428597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:19.428597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:19.428597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:19.428597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:19.428597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:19.428597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:19.428597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:19.428597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:19.428597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:19.428597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:19.428597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:19.428597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:19.429459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:19.429459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:19.429459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:19.429459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:19.429459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:19.429459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:19.429459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:19.429459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:19.429459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:19.429459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:19.429459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:19.429459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:19.429459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:19.429459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:19.429459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:19.429459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:19.430231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:19.430231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:19.430231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:19.430231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:19.430231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:19.430231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:19.430231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:19.430231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:19.430231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:19.430231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:19.430231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:47:19.430231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:19.434620 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 95/186] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:47:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:19.435978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0b5ac-6b05-49d6-a7e8-abee17c71f40 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:19.436277 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0b5ac-6b05-49d6-a7e8-abee17c71f40 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:19.436559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff0b5ac-6b05-49d6-a7e8-abee17c71f40 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:19.437846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 96/187] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:19.438711 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0b5ac-6b05-49d6-a7e8-abee17c71f40 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:19.448255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163f8b0-0435-45a1-a424-edbef5b9396a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:19.448908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163f8b0-0435-45a1-a424-edbef5b9396a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:19.449197 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163f8b0-0435-45a1-a424-edbef5b9396a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:19.449478 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0163f8b0-0435-45a1-a424-edbef5b9396a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:19.490972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:19.516409 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163f8b0-0435-45a1-a424-edbef5b9396a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:19.517816 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0163f8b0-0435-45a1-a424-edbef5b9396a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:19.532534 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:19.540279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff0b5ac-6b05-49d6-a7e8-abee17c71f40 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:19.586394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e07ac-233e-4b85-ae1b-b730e5157f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:19.587165 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e07ac-233e-4b85-ae1b-b730e5157f6a None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:19.587455 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e07ac-233e-4b85-ae1b-b730e5157f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:19.587735 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68e07ac-233e-4b85-ae1b-b730e5157f6a None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:19.588446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e07ac-233e-4b85-ae1b-b730e5157f6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:19.589905 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68e07ac-233e-4b85-ae1b-b730e5157f6a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:19.608859 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 97/188] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 22:47:19 2022] POST /identity/v3/users => generated 297 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:47:19.609658 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e07ac-233e-4b85-ae1b-b730e5157f6a None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:19.629718 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 92/189] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:19.631536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163f8b0-0435-45a1-a424-edbef5b9396a None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:19.718034 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:19.733397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68e07ac-233e-4b85-ae1b-b730e5157f6a None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:19.747216 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0163f8b0-0435-45a1-a424-edbef5b9396a None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:19.770023 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8821fd38-8046-4089-ace5-90f1837e5e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:19.770528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821fd38-8046-4089-ace5-90f1837e5e92 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:19.770847 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821fd38-8046-4089-ace5-90f1837e5e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:19.771833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8821fd38-8046-4089-ace5-90f1837e5e92 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:19.771935 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821fd38-8046-4089-ace5-90f1837e5e92 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:19.772238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821fd38-8046-4089-ace5-90f1837e5e92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:19.773626 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8821fd38-8046-4089-ace5-90f1837e5e92 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:19.783690 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 98/190] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:47:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:19.785070 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb9fd3-2821-4d92-b3f6-bccfe0157685 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:19.785367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb9fd3-2821-4d92-b3f6-bccfe0157685 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:19.785647 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceb9fd3-2821-4d92-b3f6-bccfe0157685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:19.791198 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe4aab7-f692-440e-b511-3615eb336852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:19.792260 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe4aab7-f692-440e-b511-3615eb336852 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:19.792260 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe4aab7-f692-440e-b511-3615eb336852 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:19.792510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe4aab7-f692-440e-b511-3615eb336852 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:19.793208 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe4aab7-f692-440e-b511-3615eb336852 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:19.793523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe4aab7-f692-440e-b511-3615eb336852 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:19.794980 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe4aab7-f692-440e-b511-3615eb336852 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:19.806327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 93/191] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:19.806665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe4aab7-f692-440e-b511-3615eb336852 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:19.812094 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-0ceb9fd3-2821-4d92-b3f6-bccfe0157685 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:19.813443 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-0ceb9fd3-2821-4d92-b3f6-bccfe0157685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:19.823955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:19.910482 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 99/192] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:19.911499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb9fd3-2821-4d92-b3f6-bccfe0157685 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:19.948724 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe4aab7-f692-440e-b511-3615eb336852 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:19.988118 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbe62a-fa95-4ef4-801a-93966d18995a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:19.989945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbe62a-fa95-4ef4-801a-93966d18995a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:19.990224 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbe62a-fa95-4ef4-801a-93966d18995a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:19.990525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dbe62a-fa95-4ef4-801a-93966d18995a None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:19.991799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbe62a-fa95-4ef4-801a-93966d18995a None admin] RBAC: Authorizing `identity:get_project(project_id=9bdf56f0db064649a642e4d566966738)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:19.993484 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dbe62a-fa95-4ef4-801a-93966d18995a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:19.995258 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 94/193] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:19 2022] GET /identity/v3/projects/9bdf56f0db064649a642e4d566966738 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:19.996036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbe62a-fa95-4ef4-801a-93966d18995a None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:20.000341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:20.030669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceb9fd3-2821-4d92-b3f6-bccfe0157685 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:20.075330 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2934b5-3e5a-4815-9261-951e27b64750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:20.076217 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2934b5-3e5a-4815-9261-951e27b64750 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:20.076501 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2934b5-3e5a-4815-9261-951e27b64750 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:20.076778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2934b5-3e5a-4815-9261-951e27b64750 None admin] PATH_INFO: `/v3/users/7e5f84f2dc5942378b26b2db6aa0215f` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:20.098350 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2934b5-3e5a-4815-9261-951e27b64750 None admin] RBAC: Authorizing `identity:get_user(user_id=7e5f84f2dc5942378b26b2db6aa0215f)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:20.100853 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2934b5-3e5a-4815-9261-951e27b64750 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:20.103522 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 100/194] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:19 2022] GET /identity/v3/users/7e5f84f2dc5942378b26b2db6aa0215f => generated 283 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:20.105190 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2934b5-3e5a-4815-9261-951e27b64750 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:20.109020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:20.112323 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dbe62a-fa95-4ef4-801a-93966d18995a None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:20.151009 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5118abc-ecfd-40fc-9254-7022226338a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:20.152535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5118abc-ecfd-40fc-9254-7022226338a4 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:20.153109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5118abc-ecfd-40fc-9254-7022226338a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:20.153377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5118abc-ecfd-40fc-9254-7022226338a4 None admin] PATH_INFO: `/v3/users/7e5f84f2dc5942378b26b2db6aa0215f` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:20.154651 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5118abc-ecfd-40fc-9254-7022226338a4 None admin] RBAC: Authorizing `identity:get_user(user_id=7e5f84f2dc5942378b26b2db6aa0215f)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:20.156132 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5118abc-ecfd-40fc-9254-7022226338a4 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:20.157958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 95/195] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:19 2022] GET /identity/v3/users/7e5f84f2dc5942378b26b2db6aa0215f => generated 283 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:20.162699 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5118abc-ecfd-40fc-9254-7022226338a4 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:20.253369 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:20.260302 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2934b5-3e5a-4815-9261-951e27b64750 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:20.290944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5118abc-ecfd-40fc-9254-7022226338a4 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:20.296551 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdbcd1-7deb-44d4-82c4-e3e27f24de2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:20.297530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdbcd1-7deb-44d4-82c4-e3e27f24de2c None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:20.297911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdbcd1-7deb-44d4-82c4-e3e27f24de2c None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:20.298278 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbdbcd1-7deb-44d4-82c4-e3e27f24de2c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:20.309080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbdbcd1-7deb-44d4-82c4-e3e27f24de2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:20.316974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-fcbdbcd1-7deb-44d4-82c4-e3e27f24de2c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 22:47:20.316974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:20.316974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:20.316974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:20.316974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:20.316974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:20.316974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:20.316974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:20.316974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:20.316974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:20.316974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:20.316974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:20.316974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:20.316974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:20.316974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:20.316974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:20.317972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:20.317972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:20.317972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:20.317972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:20.317972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:20.317972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:20.317972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:20.317972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:20.317972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:20.317972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:20.317972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:20.317972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:20.317972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:20.317972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:20.317972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:20.317972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:20.319337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:20.319337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:20.319337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:20.319337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:20.319337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:20.319337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:20.319337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:20.319337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:20.319337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:20.319337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:20.319337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 22:47:20.319337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:20.323049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 101/196] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:20.324035 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdbcd1-7deb-44d4-82c4-e3e27f24de2c None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:20.335018 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d349bec-07e4-409f-9c9c-77c41c79b8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:20.336074 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d349bec-07e4-409f-9c9c-77c41c79b8f7 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:20.336537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d349bec-07e4-409f-9c9c-77c41c79b8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:20.336991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d349bec-07e4-409f-9c9c-77c41c79b8f7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:20.338151 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d349bec-07e4-409f-9c9c-77c41c79b8f7 None admin] RBAC: Adding query filter params (role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.project.id=9bdf56f0db064649a642e4d566966738, user.id=1288d3a7956c42f7ad18e2e126c7dd07) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:20.338703 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d349bec-07e4-409f-9c9c-77c41c79b8f7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:20.340552 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d349bec-07e4-409f-9c9c-77c41c79b8f7 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:20.347524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 96/197] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:47:20 2022] GET /identity/v3/role_assignments?user.id=1288d3a7956c42f7ad18e2e126c7dd07&scope.project.id=9bdf56f0db064649a642e4d566966738&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:20.348618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d349bec-07e4-409f-9c9c-77c41c79b8f7 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:20.409754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:20.476189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d349bec-07e4-409f-9c9c-77c41c79b8f7 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:20.482845 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbdbcd1-7deb-44d4-82c4-e3e27f24de2c None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:20.536624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde04c20-6f38-4ba9-aaec-4b56c1d4fedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:20.539128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde04c20-6f38-4ba9-aaec-4b56c1d4fedc None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:20.540738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde04c20-6f38-4ba9-aaec-4b56c1d4fedc None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:20.541296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde04c20-6f38-4ba9-aaec-4b56c1d4fedc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:20.549682 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:20.562704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bc89f-2cf1-4806-b5d4-e880520e577a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:20.564058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bc89f-2cf1-4806-b5d4-e880520e577a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:20.564528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bc89f-2cf1-4806-b5d4-e880520e577a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:20.565337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583bc89f-2cf1-4806-b5d4-e880520e577a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:20.663828 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde04c20-6f38-4ba9-aaec-4b56c1d4fedc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:20.665846 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde04c20-6f38-4ba9-aaec-4b56c1d4fedc None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:20.689354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583bc89f-2cf1-4806-b5d4-e880520e577a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:20.690898 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583bc89f-2cf1-4806-b5d4-e880520e577a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:20.762628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:20.811075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 97/198] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:20.812271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde04c20-6f38-4ba9-aaec-4b56c1d4fedc None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:20.816356 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 102/199] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:20.819406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bc89f-2cf1-4806-b5d4-e880520e577a None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:20.858082 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:20.938125 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583bc89f-2cf1-4806-b5d4-e880520e577a None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:20.950561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde04c20-6f38-4ba9-aaec-4b56c1d4fedc None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:20.982509 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520a414-abab-4051-a640-625147ce385f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:20.984330 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520a414-abab-4051-a640-625147ce385f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:20.984858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520a414-abab-4051-a640-625147ce385f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:20.985387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520a414-abab-4051-a640-625147ce385f None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:20.987189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520a414-abab-4051-a640-625147ce385f None admin] RBAC: Authorizing `identity:get_project(project_id=f39c31704e364effbc1f39d76b609302)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:20.989696 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520a414-abab-4051-a640-625147ce385f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:20.993118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 103/200] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:20 2022] GET /identity/v3/projects/f39c31704e364effbc1f39d76b609302 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:20.994114 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520a414-abab-4051-a640-625147ce385f None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:20.998781 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4bcda-8529-47f2-a400-c5a69e5ca338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:20.999487 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4bcda-8529-47f2-a400-c5a69e5ca338 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:20.999770 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4bcda-8529-47f2-a400-c5a69e5ca338 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:21.000047 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa4bcda-8529-47f2-a400-c5a69e5ca338 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:21.000802 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4bcda-8529-47f2-a400-c5a69e5ca338 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:21.001111 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4bcda-8529-47f2-a400-c5a69e5ca338 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:21.002486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa4bcda-8529-47f2-a400-c5a69e5ca338 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:21.022764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 98/201] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:21.024499 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4bcda-8529-47f2-a400-c5a69e5ca338 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:21.039760 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:21.136312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520a414-abab-4051-a640-625147ce385f None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:21.140948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:21.146883 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa4bcda-8529-47f2-a400-c5a69e5ca338 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:21.181362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d20dd-3ed1-4774-b729-74408297a9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:21.182538 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d20dd-3ed1-4774-b729-74408297a9bb None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:21.183044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d20dd-3ed1-4774-b729-74408297a9bb None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:21.183528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206d20dd-3ed1-4774-b729-74408297a9bb None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:21.184410 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe9a9d-6e7f-4510-9add-8d33f8f5cba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:21.185064 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe9a9d-6e7f-4510-9add-8d33f8f5cba4 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:21.185322 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d20dd-3ed1-4774-b729-74408297a9bb None admin] RBAC: Authorizing `identity:get_project(project_id=f39c31704e364effbc1f39d76b609302)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:21.185421 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe9a9d-6e7f-4510-9add-8d33f8f5cba4 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:21.185614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe9a9d-6e7f-4510-9add-8d33f8f5cba4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:21.186441 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe9a9d-6e7f-4510-9add-8d33f8f5cba4 None admin] RBAC: Adding query filter params (role.id=bda1c48ca2534bc6b2ca4f3ff479b143, scope.project.id=f39c31704e364effbc1f39d76b609302, user.id=7e5f84f2dc5942378b26b2db6aa0215f) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:21.186785 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe9a9d-6e7f-4510-9add-8d33f8f5cba4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:21.187651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206d20dd-3ed1-4774-b729-74408297a9bb None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:21.188216 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe9a9d-6e7f-4510-9add-8d33f8f5cba4 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:21.189777 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 104/202] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:20 2022] GET /identity/v3/projects/f39c31704e364effbc1f39d76b609302 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:21.190924 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d20dd-3ed1-4774-b729-74408297a9bb None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:21.199294 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 99/203] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:47:21 2022] GET /identity/v3/role_assignments?user.id=7e5f84f2dc5942378b26b2db6aa0215f&scope.project.id=f39c31704e364effbc1f39d76b609302&role.id=bda1c48ca2534bc6b2ca4f3ff479b143 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:21.200186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe9a9d-6e7f-4510-9add-8d33f8f5cba4 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:21.330403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:21.338204 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206d20dd-3ed1-4774-b729-74408297a9bb None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:21.384753 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206e1e4-0e27-40a5-90ab-e352644a4d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:21.386025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206e1e4-0e27-40a5-90ab-e352644a4d3f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:21.388049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206e1e4-0e27-40a5-90ab-e352644a4d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:21.388561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c206e1e4-0e27-40a5-90ab-e352644a4d3f None admin] PATH_INFO: `/v3/users/1e872aac4d734b22b3ef7ea653686c96` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:21.391157 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206e1e4-0e27-40a5-90ab-e352644a4d3f None admin] RBAC: Authorizing `identity:get_user(user_id=1e872aac4d734b22b3ef7ea653686c96)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:21.394384 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c206e1e4-0e27-40a5-90ab-e352644a4d3f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:21.403231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 105/204] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:21 2022] GET /identity/v3/users/1e872aac4d734b22b3ef7ea653686c96 => generated 289 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:21.406755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206e1e4-0e27-40a5-90ab-e352644a4d3f None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:21.412562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe9a9d-6e7f-4510-9add-8d33f8f5cba4 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:21.474705 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd2cc8-81e1-4602-bc59-aa7074704d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:21.475751 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd2cc8-81e1-4602-bc59-aa7074704d86 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:21.476230 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd2cc8-81e1-4602-bc59-aa7074704d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:21.477125 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcd2cc8-81e1-4602-bc59-aa7074704d86 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:21.478187 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd2cc8-81e1-4602-bc59-aa7074704d86 None admin] RBAC: Adding query filter params (role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.project.id=f39c31704e364effbc1f39d76b609302, user.id=7e5f84f2dc5942378b26b2db6aa0215f) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:21.478723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd2cc8-81e1-4602-bc59-aa7074704d86 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:21.480377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcd2cc8-81e1-4602-bc59-aa7074704d86 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:21.487766 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 100/205] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:47:21 2022] GET /identity/v3/role_assignments?user.id=7e5f84f2dc5942378b26b2db6aa0215f&scope.project.id=f39c31704e364effbc1f39d76b609302&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 259 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:21.594245 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c206e1e4-0e27-40a5-90ab-e352644a4d3f None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:21.647011 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198963d5-f4e9-43a4-b23c-d8e5c269f4a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:21.647624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198963d5-f4e9-43a4-b23c-d8e5c269f4a1 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:21.647911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198963d5-f4e9-43a4-b23c-d8e5c269f4a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:21.648192 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198963d5-f4e9-43a4-b23c-d8e5c269f4a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:21.690313 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:21.730448 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198963d5-f4e9-43a4-b23c-d8e5c269f4a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:21.732053 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198963d5-f4e9-43a4-b23c-d8e5c269f4a1 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:21.859911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 106/206] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:21.865796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd2cc8-81e1-4602-bc59-aa7074704d86 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:21.905376 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:21.983671 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901bc68d-b536-43be-985b-bf026ec6c9d3 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:21.984447 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901bc68d-b536-43be-985b-bf026ec6c9d3 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:21.984929 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901bc68d-b536-43be-985b-bf026ec6c9d3 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:21.986547 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 107/207] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:47:21.994146 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcd2cc8-81e1-4602-bc59-aa7074704d86 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:22.004405 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e6ed9-9c87-4e97-84d3-e8f32fbfdba4 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:22.004975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e6ed9-9c87-4e97-84d3-e8f32fbfdba4 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:22.005445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e6ed9-9c87-4e97-84d3-e8f32fbfdba4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:22.028538 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-6b1e6ed9-9c87-4e97-84d3-e8f32fbfdba4 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:22.030828 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6b1e6ed9-9c87-4e97-84d3-e8f32fbfdba4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:22.066394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f210bc0-c811-415a-bd25-40ba10723cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:22.067664 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f210bc0-c811-415a-bd25-40ba10723cbd None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:22.068733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f210bc0-c811-415a-bd25-40ba10723cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:22.069203 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f210bc0-c811-415a-bd25-40ba10723cbd None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:22.071105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f210bc0-c811-415a-bd25-40ba10723cbd None admin] RBAC: Authorizing `identity:get_project(project_id=9bdf56f0db064649a642e4d566966738)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:22.072819 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f210bc0-c811-415a-bd25-40ba10723cbd None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:22.074793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 101/208] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:21 2022] GET /identity/v3/projects/9bdf56f0db064649a642e4d566966738 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:22.081731 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f210bc0-c811-415a-bd25-40ba10723cbd None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:22.150961 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:22.165257 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 108/209] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:22.213699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e6ed9-9c87-4e97-84d3-e8f32fbfdba4 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:22.257111 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:22.281797 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f210bc0-c811-415a-bd25-40ba10723cbd None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:22.344121 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e6ed9-9c87-4e97-84d3-e8f32fbfdba4 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:22.362289 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2615ca8-1c01-4036-9dae-bf6b72df4045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:22.363254 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2615ca8-1c01-4036-9dae-bf6b72df4045 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:22.363548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2615ca8-1c01-4036-9dae-bf6b72df4045 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:22.363834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2615ca8-1c01-4036-9dae-bf6b72df4045 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:22.365738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2615ca8-1c01-4036-9dae-bf6b72df4045 None admin] RBAC: Adding query filter params (role.id=8eefae9314f544de97883fa2b10a2046, scope.project.id=9bdf56f0db064649a642e4d566966738, user.id=1e872aac4d734b22b3ef7ea653686c96) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:22.366063 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2615ca8-1c01-4036-9dae-bf6b72df4045 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:22.368088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2615ca8-1c01-4036-9dae-bf6b72df4045 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:22.375981 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 102/210] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:47:22 2022] GET /identity/v3/role_assignments?user.id=1e872aac4d734b22b3ef7ea653686c96&scope.project.id=9bdf56f0db064649a642e4d566966738&role.id=8eefae9314f544de97883fa2b10a2046 => generated 608 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:22.378109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c5fdd-520d-4eed-a9cb-550356c4a446 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:22.378412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c5fdd-520d-4eed-a9cb-550356c4a446 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:22.378728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c44c5fdd-520d-4eed-a9cb-550356c4a446 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:22.381149 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 103/211] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:22.385633 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13244f6-70ea-4ed5-ba10-abbbc6d0d23c None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:22.386152 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13244f6-70ea-4ed5-ba10-abbbc6d0d23c None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:22.386668 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13244f6-70ea-4ed5-ba10-abbbc6d0d23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:22.399947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab8767-3230-4134-87a7-61ce5f8e89fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:22.402399 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab8767-3230-4134-87a7-61ce5f8e89fd None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:22.402975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab8767-3230-4134-87a7-61ce5f8e89fd None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:22.403449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eab8767-3230-4134-87a7-61ce5f8e89fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:22.424738 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eab8767-3230-4134-87a7-61ce5f8e89fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:22.426634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-d13244f6-70ea-4ed5-ba10-abbbc6d0d23c None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:22.428391 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-d13244f6-70ea-4ed5-ba10-abbbc6d0d23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:22.434469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-4eab8767-3230-4134-87a7-61ce5f8e89fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:22.434469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:22.434469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:22.434469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:22.434469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:22.434469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:22.434469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:22.434469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:22.434469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:22.434469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:22.434469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:22.434469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:22.434469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:22.434469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:22.434469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:22.434469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:22.435413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:22.435413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:22.435413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:22.435413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:22.435413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:22.435413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:22.435413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:22.435413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:22.435413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:22.435413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:22.435413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:22.435413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:22.435413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:22.435413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:22.435413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:22.435413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:22.436185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:22.436185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:22.436185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:22.436185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:22.436185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:22.436185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:22.436185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:22.436185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:22.436185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:22.436185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:22.436185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:22.436185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:22.439118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 109/212] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:47:22.449327 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab8767-3230-4134-87a7-61ce5f8e89fd None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:22.518619 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 104/213] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:47:22.542100 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13244f6-70ea-4ed5-ba10-abbbc6d0d23c None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:22.571005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:22.632503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eab8767-3230-4134-87a7-61ce5f8e89fd None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:22.660216 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:22.673749 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13244f6-70ea-4ed5-ba10-abbbc6d0d23c None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:22.680631 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f10fd2-e195-4f58-abe8-01106ba1d932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:22.681502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f10fd2-e195-4f58-abe8-01106ba1d932 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:22.681927 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f10fd2-e195-4f58-abe8-01106ba1d932 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:22.682866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f10fd2-e195-4f58-abe8-01106ba1d932 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:22.683893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f10fd2-e195-4f58-abe8-01106ba1d932 None admin] RBAC: Adding query filter params (name=member) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:22.684343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f10fd2-e195-4f58-abe8-01106ba1d932 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:22.686734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f10fd2-e195-4f58-abe8-01106ba1d932 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:22.698766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 110/214] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:22.706633 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f10fd2-e195-4f58-abe8-01106ba1d932 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:22.719891 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b6ef0-8364-46ad-aab7-98e7eb518cea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:22.721131 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b6ef0-8364-46ad-aab7-98e7eb518cea None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:22.721313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b6ef0-8364-46ad-aab7-98e7eb518cea None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:22.721691 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78b6ef0-8364-46ad-aab7-98e7eb518cea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:22.723167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b6ef0-8364-46ad-aab7-98e7eb518cea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:22.724804 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78b6ef0-8364-46ad-aab7-98e7eb518cea None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:22.727003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 105/215] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:47:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:22.735378 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b6ef0-8364-46ad-aab7-98e7eb518cea None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:22.835245 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f10fd2-e195-4f58-abe8-01106ba1d932 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:22.854007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:22.870942 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78b6ef0-8364-46ad-aab7-98e7eb518cea None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:22.872400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ec927-9ca6-4297-a305-6592507943d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:22.873364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ec927-9ca6-4297-a305-6592507943d9 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:22.873661 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ec927-9ca6-4297-a305-6592507943d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:22.873941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33ec927-9ca6-4297-a305-6592507943d9 None admin] PATH_INFO: `/v3/users/2be5740ae47448b99ba83deae888b58c` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:22.892802 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ec927-9ca6-4297-a305-6592507943d9 None admin] RBAC: Authorizing `identity:get_user(user_id=2be5740ae47448b99ba83deae888b58c)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:22.894312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33ec927-9ca6-4297-a305-6592507943d9 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:22.896084 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 111/216] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:22 2022] GET /identity/v3/users/2be5740ae47448b99ba83deae888b58c => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:22.901962 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ec927-9ca6-4297-a305-6592507943d9 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:22.918420 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da4397-609d-4f7a-aec9-528a63e06d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:22.919547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da4397-609d-4f7a-aec9-528a63e06d4b None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:22.919974 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da4397-609d-4f7a-aec9-528a63e06d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:22.920733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18da4397-609d-4f7a-aec9-528a63e06d4b None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:22.921963 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da4397-609d-4f7a-aec9-528a63e06d4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:22.923688 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18da4397-609d-4f7a-aec9-528a63e06d4b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:22.953505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 106/217] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 22:47:22 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:47:23.020445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33ec927-9ca6-4297-a305-6592507943d9 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:23.058949 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbb66e-feb9-4346-a443-44add3744d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:23.059628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbb66e-feb9-4346-a443-44add3744d1a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:23.059919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbb66e-feb9-4346-a443-44add3744d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:23.060209 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dcbb66e-feb9-4346-a443-44add3744d1a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:23.113137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:23.136894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcbb66e-feb9-4346-a443-44add3744d1a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:23.138473 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dcbb66e-feb9-4346-a443-44add3744d1a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:23.250947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 112/218] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 22:47:23.257429 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da4397-609d-4f7a-aec9-528a63e06d4b None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:23.279476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:23.382510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18da4397-609d-4f7a-aec9-528a63e06d4b None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:23.419422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96f2a1-f48f-4f6e-a973-bbe8bcc51f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:23.420675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96f2a1-f48f-4f6e-a973-bbe8bcc51f69 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:23.421190 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96f2a1-f48f-4f6e-a973-bbe8bcc51f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:23.421659 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96f2a1-f48f-4f6e-a973-bbe8bcc51f69 None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:23.423128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96f2a1-f48f-4f6e-a973-bbe8bcc51f69 None admin] RBAC: Authorizing `identity:get_project(project_id=f39c31704e364effbc1f39d76b609302)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:23.424774 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96f2a1-f48f-4f6e-a973-bbe8bcc51f69 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:23.426763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 107/219] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:23 2022] GET /identity/v3/projects/f39c31704e364effbc1f39d76b609302 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:23.432902 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbb66e-feb9-4346-a443-44add3744d1a None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:23.545270 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:23.580921 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dcbb66e-feb9-4346-a443-44add3744d1a None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:23.631535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f4d5c7-625c-4c2b-919c-c602ac352f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:23.632193 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4d5c7-625c-4c2b-919c-c602ac352f8f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:23.632480 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4d5c7-625c-4c2b-919c-c602ac352f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:23.632758 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f4d5c7-625c-4c2b-919c-c602ac352f8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:23.633592 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4d5c7-625c-4c2b-919c-c602ac352f8f None admin] RBAC: Adding query filter params (role.id=bff4f694fb57451e9a0783dc014187f2, scope.project.id=f39c31704e364effbc1f39d76b609302, user.id=2be5740ae47448b99ba83deae888b58c) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:23.633906 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4d5c7-625c-4c2b-919c-c602ac352f8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:23.635380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f4d5c7-625c-4c2b-919c-c602ac352f8f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:23.646950 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 113/220] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:47:23 2022] GET /identity/v3/role_assignments?user.id=2be5740ae47448b99ba83deae888b58c&scope.project.id=f39c31704e364effbc1f39d76b609302&role.id=bff4f694fb57451e9a0783dc014187f2 => generated 259 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:23.907979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a3629-b6ca-4e8d-9d99-72cc42453e8b None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:23.908531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a3629-b6ca-4e8d-9d99-72cc42453e8b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:23.908531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1a3629-b6ca-4e8d-9d99-72cc42453e8b None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:23.910243 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 108/221] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:23.915475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7374f8-2b6a-42b1-a9d0-0fa489e04140 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:23.915768 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7374f8-2b6a-42b1-a9d0-0fa489e04140 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:23.916055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7374f8-2b6a-42b1-a9d0-0fa489e04140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:23.940792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-7a7374f8-2b6a-42b1-a9d0-0fa489e04140 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:23.941985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-7a7374f8-2b6a-42b1-a9d0-0fa489e04140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:23.979978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acb46e-03fe-4d3a-8643-e884f6726378 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:23.980603 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acb46e-03fe-4d3a-8643-e884f6726378 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:23.981085 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4acb46e-03fe-4d3a-8643-e884f6726378 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:23.982722 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 109/222] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:23.987922 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffa19b-7023-44dd-b975-a3a36bc132f3 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:23.988444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffa19b-7023-44dd-b975-a3a36bc132f3 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:23.988939 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ffa19b-7023-44dd-b975-a3a36bc132f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:24.010397 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-11ffa19b-7023-44dd-b975-a3a36bc132f3 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:24.012308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-11ffa19b-7023-44dd-b975-a3a36bc132f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:24.018573 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 114/223] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:24.058696 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7374f8-2b6a-42b1-a9d0-0fa489e04140 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:24.144991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 110/224] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:47:24.180006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:24.220061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7374f8-2b6a-42b1-a9d0-0fa489e04140 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:24.225957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffa19b-7023-44dd-b975-a3a36bc132f3 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:24.250936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:24.259182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa87ad9-77c7-4d31-8487-a5363d0da37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:24.260003 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa87ad9-77c7-4d31-8487-a5363d0da37a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:24.260289 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa87ad9-77c7-4d31-8487-a5363d0da37a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:24.260568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa87ad9-77c7-4d31-8487-a5363d0da37a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:24.271749 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa87ad9-77c7-4d31-8487-a5363d0da37a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:24.280429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-9fa87ad9-77c7-4d31-8487-a5363d0da37a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:24.280429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:24.280429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:24.280429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:24.280429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:24.280429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:24.280429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:24.280429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:24.280429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:24.280429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:24.280429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:24.280429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:24.280429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:24.280429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:24.280429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:24.280429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:24.281283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:24.281283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:24.281283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:24.281283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:24.281283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:24.281283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:24.281283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:24.281283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:24.281283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:24.281283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:24.281283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:24.281283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:24.281283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:24.281283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:24.281283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:24.281283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:24.282047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:24.282047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:24.282047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:24.282047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:24.282047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:24.282047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:24.282047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:24.282047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:24.282047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:24.282047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:24.282047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:24.282047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:24.282047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 115/225] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:47:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:47:24.289638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa87ad9-77c7-4d31-8487-a5363d0da37a None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:24.400888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa87ad9-77c7-4d31-8487-a5363d0da37a None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:24.436078 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:24.443752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10494666-774d-4435-9480-62adfcb59ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:24.444777 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10494666-774d-4435-9480-62adfcb59ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:24.445341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10494666-774d-4435-9480-62adfcb59ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:24.446450 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10494666-774d-4435-9480-62adfcb59ed1 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:24.447400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10494666-774d-4435-9480-62adfcb59ed1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:24.447863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10494666-774d-4435-9480-62adfcb59ed1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:24.449435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10494666-774d-4435-9480-62adfcb59ed1 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:24.458287 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 116/226] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:47:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:24.464819 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10494666-774d-4435-9480-62adfcb59ed1 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:24.516704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ffa19b-7023-44dd-b975-a3a36bc132f3 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:24.573060 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10494666-774d-4435-9480-62adfcb59ed1 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:24.586809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4b918-9af8-4312-9bc7-b566a48bfec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:24.587855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4b918-9af8-4312-9bc7-b566a48bfec1 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:24.588282 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4b918-9af8-4312-9bc7-b566a48bfec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:24.588700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa4b918-9af8-4312-9bc7-b566a48bfec1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:24.608130 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa4b918-9af8-4312-9bc7-b566a48bfec1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:24.616957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-0aa4b918-9af8-4312-9bc7-b566a48bfec1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:47:24.616957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:24.616957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:24.616957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:24.616957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:24.616957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:24.616957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:24.616957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:24.616957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:24.616957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:24.616957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:24.616957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:24.616957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:24.616957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:24.616957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:24.616957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:24.617814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:24.617814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:24.617814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:24.617814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:24.617814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:24.617814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:24.617814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:24.617814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:24.617814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:24.617814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:24.617814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:24.617814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:24.617814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:24.617814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:24.617814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:24.617814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:24.618626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:24.618626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:24.618626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:24.618626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:24.618626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:24.618626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:24.618626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:24.618626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:24.618626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:24.618626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:24.618626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:47:24.618626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:24.621303 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 111/227] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:47:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 396 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:24.627446 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4b918-9af8-4312-9bc7-b566a48bfec1 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:24.653863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470aee3-caeb-4323-b007-d845beca825d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:24.655692 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470aee3-caeb-4323-b007-d845beca825d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:24.656892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470aee3-caeb-4323-b007-d845beca825d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:24.657578 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3470aee3-caeb-4323-b007-d845beca825d None admin] PATH_INFO: `/v3/users/7e5f84f2dc5942378b26b2db6aa0215f` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:24.660330 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470aee3-caeb-4323-b007-d845beca825d None admin] RBAC: Authorizing `identity:get_user(user_id=7e5f84f2dc5942378b26b2db6aa0215f)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:24.663144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3470aee3-caeb-4323-b007-d845beca825d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:24.667256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 117/228] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:24 2022] GET /identity/v3/users/7e5f84f2dc5942378b26b2db6aa0215f => generated 283 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:24.673334 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470aee3-caeb-4323-b007-d845beca825d None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:24.755300 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:24.764515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:24.800242 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3470aee3-caeb-4323-b007-d845beca825d None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:24.836799 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc7745-ffc0-4019-a061-0c381fddce13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:24.838899 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc7745-ffc0-4019-a061-0c381fddce13 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:24.839981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc7745-ffc0-4019-a061-0c381fddce13 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:24.840654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc7745-ffc0-4019-a061-0c381fddce13 None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:24.842653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa4b918-9af8-4312-9bc7-b566a48bfec1 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:24.847960 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc7745-ffc0-4019-a061-0c381fddce13 None admin] RBAC: Authorizing `identity:get_project(project_id=f39c31704e364effbc1f39d76b609302)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:24.849424 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bc7745-ffc0-4019-a061-0c381fddce13 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:24.857028 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 118/229] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:24 2022] GET /identity/v3/projects/f39c31704e364effbc1f39d76b609302 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:24.858165 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc7745-ffc0-4019-a061-0c381fddce13 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:24.881343 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd136b8-93fd-4e63-8077-86529f223bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:24.882432 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd136b8-93fd-4e63-8077-86529f223bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:24.882966 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd136b8-93fd-4e63-8077-86529f223bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:24.883492 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd136b8-93fd-4e63-8077-86529f223bd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:24.884451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd136b8-93fd-4e63-8077-86529f223bd4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:24.884955 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd136b8-93fd-4e63-8077-86529f223bd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:24.886556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd136b8-93fd-4e63-8077-86529f223bd4 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:24.896486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 112/230] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:24.902279 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd136b8-93fd-4e63-8077-86529f223bd4 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:25.059957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd136b8-93fd-4e63-8077-86529f223bd4 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:25.097184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bc7745-ffc0-4019-a061-0c381fddce13 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:25.097855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:25.119337 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cccc20-8ede-4bd0-9e03-203e72e36be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:25.120250 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cccc20-8ede-4bd0-9e03-203e72e36be2 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:25.120538 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cccc20-8ede-4bd0-9e03-203e72e36be2 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:25.120813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3cccc20-8ede-4bd0-9e03-203e72e36be2 None admin] PATH_INFO: `/v3/users/7e5f84f2dc5942378b26b2db6aa0215f` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:25.127388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cccc20-8ede-4bd0-9e03-203e72e36be2 None admin] RBAC: Authorizing `identity:get_user(user_id=7e5f84f2dc5942378b26b2db6aa0215f)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:25.128855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3cccc20-8ede-4bd0-9e03-203e72e36be2 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:25.130547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 113/231] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:24 2022] GET /identity/v3/users/7e5f84f2dc5942378b26b2db6aa0215f => generated 283 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:25.136002 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fdbb59-9366-4ef3-8777-145b0a788ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:25.136907 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdbb59-9366-4ef3-8777-145b0a788ea7 None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:25.137191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdbb59-9366-4ef3-8777-145b0a788ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:25.137466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fdbb59-9366-4ef3-8777-145b0a788ea7 None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302/users/7e5f84f2dc5942378b26b2db6aa0215f/roles/bda1c48ca2534bc6b2ca4f3ff479b143` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:25.139495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cccc20-8ede-4bd0-9e03-203e72e36be2 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:25.139719 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fdbb59-9366-4ef3-8777-145b0a788ea7 None admin] RBAC: Authorizing `identity:create_grant(project_id=f39c31704e364effbc1f39d76b609302, user_id=7e5f84f2dc5942378b26b2db6aa0215f, role_id=bda1c48ca2534bc6b2ca4f3ff479b143)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:25.141260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fdbb59-9366-4ef3-8777-145b0a788ea7 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:25.145633 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:25.160445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 119/232] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:47:24 2022] PUT /identity/v3/projects/f39c31704e364effbc1f39d76b609302/users/7e5f84f2dc5942378b26b2db6aa0215f/roles/bda1c48ca2534bc6b2ca4f3ff479b143 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 22:47:25.306539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3cccc20-8ede-4bd0-9e03-203e72e36be2 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:25.356307 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6f5439-74ab-412e-b23e-9d9e807135bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:25.357243 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f5439-74ab-412e-b23e-9d9e807135bf None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:25.357537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f5439-74ab-412e-b23e-9d9e807135bf None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:25.357818 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f5439-74ab-412e-b23e-9d9e807135bf None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:25.359139 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f5439-74ab-412e-b23e-9d9e807135bf None admin] RBAC: Authorizing `identity:get_project(project_id=f39c31704e364effbc1f39d76b609302)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:25.360583 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6f5439-74ab-412e-b23e-9d9e807135bf None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:25.362566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 114/233] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:25 2022] GET /identity/v3/projects/f39c31704e364effbc1f39d76b609302 => generated 302 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:25.368726 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fdbb59-9366-4ef3-8777-145b0a788ea7 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:25.522910 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fdbb59-9366-4ef3-8777-145b0a788ea7 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:25.575858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052690dc-85e2-4083-accc-3baeaf4819ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:25.576772 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052690dc-85e2-4083-accc-3baeaf4819ad None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:25.577059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052690dc-85e2-4083-accc-3baeaf4819ad None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:25.577338 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052690dc-85e2-4083-accc-3baeaf4819ad None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302/users/7e5f84f2dc5942378b26b2db6aa0215f/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:25.579375 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052690dc-85e2-4083-accc-3baeaf4819ad None admin] RBAC: Authorizing `identity:create_grant(project_id=f39c31704e364effbc1f39d76b609302, user_id=7e5f84f2dc5942378b26b2db6aa0215f, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:25.580821 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052690dc-85e2-4083-accc-3baeaf4819ad None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:25.607903 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 120/234] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:47:25 2022] PUT /identity/v3/projects/f39c31704e364effbc1f39d76b609302/users/7e5f84f2dc5942378b26b2db6aa0215f/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:47:25.671328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bcd30-4085-4838-92f6-7849659eb87f None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:25.671624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bcd30-4085-4838-92f6-7849659eb87f None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:25.672087 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497bcd30-4085-4838-92f6-7849659eb87f None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:25.673563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 115/235] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:25.679527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5201d5-2219-4196-8935-35d362e67251 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:25.680062 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5201d5-2219-4196-8935-35d362e67251 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:25.680356 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5201d5-2219-4196-8935-35d362e67251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:25.701048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-5c5201d5-2219-4196-8935-35d362e67251 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:25.702379 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-5c5201d5-2219-4196-8935-35d362e67251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:25.765717 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:25.772929 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a8d79-3e36-4647-9672-77224ab9c953 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:25.773234 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a8d79-3e36-4647-9672-77224ab9c953 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:25.773493 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396a8d79-3e36-4647-9672-77224ab9c953 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:25.774965 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 116/236] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:25.797571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df2445-22de-4ad6-8af9-3c1d5db0488b None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:25.797881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df2445-22de-4ad6-8af9-3c1d5db0488b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:25.798135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df2445-22de-4ad6-8af9-3c1d5db0488b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:25.803937 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 121/237] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:25.827782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5201d5-2219-4196-8935-35d362e67251 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:25.831006 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a7df2445-22de-4ad6-8af9-3c1d5db0488b None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:25.832222 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a7df2445-22de-4ad6-8af9-3c1d5db0488b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:25.910839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:25.917626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 117/238] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jun 16 22:47:25.944253 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5201d5-2219-4196-8935-35d362e67251 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:25.947316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df2445-22de-4ad6-8af9-3c1d5db0488b None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:25.987277 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e876f-5cc0-417c-9234-cfc846ffa7e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:25.988653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e876f-5cc0-417c-9234-cfc846ffa7e3 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:25.989252 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e876f-5cc0-417c-9234-cfc846ffa7e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:25.989539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8e876f-5cc0-417c-9234-cfc846ffa7e3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:26.003550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8e876f-5cc0-417c-9234-cfc846ffa7e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:26.013377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-9c8e876f-5cc0-417c-9234-cfc846ffa7e3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 22:47:26.013377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:26.013377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:26.013377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:26.013377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:26.013377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:26.013377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:26.013377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:26.013377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:26.013377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:26.013377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:26.013377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:26.013377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:26.013377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:26.013377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:26.013377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:26.015554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:26.015554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:26.015554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:26.015554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:26.015554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:26.015554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:26.015554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:26.015554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:26.015554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:26.015554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:26.015554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:26.015554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:26.015554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:26.015554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:26.015554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:26.015554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:26.017483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:26.017483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:26.017483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:26.017483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:26.017483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:26.017483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:26.017483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:26.017483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:26.017483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:26.017483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:26.017483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 22:47:26.017483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:26.017483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 122/239] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:47:26.021284 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e876f-5cc0-417c-9234-cfc846ffa7e3 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:26.091085 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:26.136355 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df2445-22de-4ad6-8af9-3c1d5db0488b None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:26.182249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8512e-05af-4585-ac72-05095220c6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:26.183896 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8512e-05af-4585-ac72-05095220c6ea None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:26.184183 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8512e-05af-4585-ac72-05095220c6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:26.184459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8512e-05af-4585-ac72-05095220c6ea None admin] PATH_INFO: `/v3/domains/default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:26.186673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8512e-05af-4585-ac72-05095220c6ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:26.189402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc8512e-05af-4585-ac72-05095220c6ea None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:26.192359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 118/240] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:47:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:26.199670 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8512e-05af-4585-ac72-05095220c6ea None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:26.225775 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8e876f-5cc0-417c-9234-cfc846ffa7e3 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:26.272220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aeaf3fc-437a-41fb-b858-1a13d4470720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:26.273191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeaf3fc-437a-41fb-b858-1a13d4470720 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:26.273649 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeaf3fc-437a-41fb-b858-1a13d4470720 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:26.274079 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aeaf3fc-437a-41fb-b858-1a13d4470720 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:26.274998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeaf3fc-437a-41fb-b858-1a13d4470720 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:26.275948 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeaf3fc-437a-41fb-b858-1a13d4470720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:26.278061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aeaf3fc-437a-41fb-b858-1a13d4470720 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:26.290618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 123/241] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:26.296486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0359da-c6db-43bd-839d-b3412575c42f None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:26.296486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0359da-c6db-43bd-839d-b3412575c42f None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:26.296486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0359da-c6db-43bd-839d-b3412575c42f None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:26.296843 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 124/242] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:26.297493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0359da-c6db-43bd-839d-b3412575c42f None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:26.322925 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:26.363213 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8512e-05af-4585-ac72-05095220c6ea None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:26.401964 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28bea0a-7b79-4788-a60a-47bcf94dbc31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:26.403775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bea0a-7b79-4788-a60a-47bcf94dbc31 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:26.404273 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bea0a-7b79-4788-a60a-47bcf94dbc31 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:26.404814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28bea0a-7b79-4788-a60a-47bcf94dbc31 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:26.406215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28bea0a-7b79-4788-a60a-47bcf94dbc31 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:26.408341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28bea0a-7b79-4788-a60a-47bcf94dbc31 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:26.429046 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:26.434328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0359da-c6db-43bd-839d-b3412575c42f None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:26.442964 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 119/243] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 22:47:26 2022] POST /identity/v3/users => generated 293 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:47:26.445218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9539bb-e822-4230-a790-61069fbea35b None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:26.445513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9539bb-e822-4230-a790-61069fbea35b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:26.445797 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9539bb-e822-4230-a790-61069fbea35b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:26.475200 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8b291-7502-4680-90f6-5f7c3da1eb63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:26.476506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8b291-7502-4680-90f6-5f7c3da1eb63 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:26.477028 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8b291-7502-4680-90f6-5f7c3da1eb63 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:26.477503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e8b291-7502-4680-90f6-5f7c3da1eb63 None admin] PATH_INFO: `/v3/users/911af5ec66ed4d1e848982b610db26f6` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:26.481797 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-2d9539bb-e822-4230-a790-61069fbea35b None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:26.482241 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-2d9539bb-e822-4230-a790-61069fbea35b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:26.494682 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e8b291-7502-4680-90f6-5f7c3da1eb63 None admin] RBAC: Authorizing `identity:get_user(user_id=911af5ec66ed4d1e848982b610db26f6)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:26.496621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e8b291-7502-4680-90f6-5f7c3da1eb63 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:26.499220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 125/244] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:26 2022] GET /identity/v3/users/911af5ec66ed4d1e848982b610db26f6 => generated 297 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:26.507784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8b291-7502-4680-90f6-5f7c3da1eb63 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:26.632495 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e8b291-7502-4680-90f6-5f7c3da1eb63 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:26.633969 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 120/245] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:26.659346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9539bb-e822-4230-a790-61069fbea35b None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:26.714772 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edac789-774e-4d0d-bf5a-84414c5ad7f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:26.715408 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edac789-774e-4d0d-bf5a-84414c5ad7f3 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:26.715654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edac789-774e-4d0d-bf5a-84414c5ad7f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:26.715938 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edac789-774e-4d0d-bf5a-84414c5ad7f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:26.735695 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:26.770774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:26.811966 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9539bb-e822-4230-a790-61069fbea35b None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:26.831219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edac789-774e-4d0d-bf5a-84414c5ad7f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:26.833401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edac789-774e-4d0d-bf5a-84414c5ad7f3 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:26.859140 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07742cef-90a5-4663-8dc6-eb3fd49cdf07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:26.860263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07742cef-90a5-4663-8dc6-eb3fd49cdf07 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:26.861170 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07742cef-90a5-4663-8dc6-eb3fd49cdf07 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:26.861654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07742cef-90a5-4663-8dc6-eb3fd49cdf07 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:26.880550 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07742cef-90a5-4663-8dc6-eb3fd49cdf07 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:26.891395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-07742cef-90a5-4663-8dc6-eb3fd49cdf07 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:26.891395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:26.891395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:26.891395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:26.891395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:26.891395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:26.891395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:26.891395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:26.891395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:26.891395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:26.891395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:26.891395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:26.891395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:26.891395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:26.891395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:26.891395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:26.892248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:26.892248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:26.892248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:26.892248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:26.892248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:26.892248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:26.892248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:26.892248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:26.892248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:26.892248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:26.892248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:26.892248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:26.892248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:26.892248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:26.892248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:26.892248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:26.895610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:26.895610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:26.895610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:26.895610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:26.895610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:26.895610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:26.895610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:26.895610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:26.895610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:26.895610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:26.895610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:26.895610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:26.895610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 121/246] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:47:26.899729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07742cef-90a5-4663-8dc6-eb3fd49cdf07 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:26.959922 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 126/247] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:26.968529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edac789-774e-4d0d-bf5a-84414c5ad7f3 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:27.026607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07742cef-90a5-4663-8dc6-eb3fd49cdf07 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:27.036852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:27.041881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:27.074025 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea0060-3854-484b-b2a3-6fc720e63b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:27.074753 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea0060-3854-484b-b2a3-6fc720e63b7e None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:27.075129 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea0060-3854-484b-b2a3-6fc720e63b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:27.075399 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea0060-3854-484b-b2a3-6fc720e63b7e None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:27.076081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea0060-3854-484b-b2a3-6fc720e63b7e None admin] RBAC: Adding query filter params (name=member) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:27.076393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea0060-3854-484b-b2a3-6fc720e63b7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:27.077778 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea0060-3854-484b-b2a3-6fc720e63b7e None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:27.088449 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 122/248] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:27.093860 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea0060-3854-484b-b2a3-6fc720e63b7e None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:27.104705 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edac789-774e-4d0d-bf5a-84414c5ad7f3 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:27.145922 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43fd7e-3df1-4a00-b4dd-cdd255b1eb9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:27.146875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43fd7e-3df1-4a00-b4dd-cdd255b1eb9e None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:27.147170 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43fd7e-3df1-4a00-b4dd-cdd255b1eb9e None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:27.147453 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb43fd7e-3df1-4a00-b4dd-cdd255b1eb9e None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:27.148716 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43fd7e-3df1-4a00-b4dd-cdd255b1eb9e None admin] RBAC: Authorizing `identity:get_project(project_id=f39c31704e364effbc1f39d76b609302)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:27.150301 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb43fd7e-3df1-4a00-b4dd-cdd255b1eb9e None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:27.152602 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 127/249] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:26 2022] GET /identity/v3/projects/f39c31704e364effbc1f39d76b609302 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:27.158938 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43fd7e-3df1-4a00-b4dd-cdd255b1eb9e None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:27.209543 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea0060-3854-484b-b2a3-6fc720e63b7e None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:27.246666 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712be543-7777-4fd1-9a62-11d6ce0fb7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:27.247556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712be543-7777-4fd1-9a62-11d6ce0fb7ca None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:27.247849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712be543-7777-4fd1-9a62-11d6ce0fb7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:27.248132 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712be543-7777-4fd1-9a62-11d6ce0fb7ca None admin] PATH_INFO: `/v3/users/2be5740ae47448b99ba83deae888b58c` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:27.249357 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712be543-7777-4fd1-9a62-11d6ce0fb7ca None admin] RBAC: Authorizing `identity:get_user(user_id=2be5740ae47448b99ba83deae888b58c)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:27.250871 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712be543-7777-4fd1-9a62-11d6ce0fb7ca None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:27.252539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 123/250] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:27 2022] GET /identity/v3/users/2be5740ae47448b99ba83deae888b58c => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:27.258577 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712be543-7777-4fd1-9a62-11d6ce0fb7ca None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:27.309148 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb43fd7e-3df1-4a00-b4dd-cdd255b1eb9e None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:27.343073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:27.350932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:27.352742 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43f81e-4245-4a8d-8338-985f1eae0d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:27.353577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43f81e-4245-4a8d-8338-985f1eae0d68 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:27.353885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43f81e-4245-4a8d-8338-985f1eae0d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:27.354159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a43f81e-4245-4a8d-8338-985f1eae0d68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:27.355112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43f81e-4245-4a8d-8338-985f1eae0d68 None admin] RBAC: Adding query filter params (role.id=8eefae9314f544de97883fa2b10a2046, scope.project.id=f39c31704e364effbc1f39d76b609302, user.id=911af5ec66ed4d1e848982b610db26f6) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:27.355435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43f81e-4245-4a8d-8338-985f1eae0d68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:27.357055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a43f81e-4245-4a8d-8338-985f1eae0d68 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:27.363931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 128/251] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:47:27 2022] GET /identity/v3/role_assignments?user.id=911af5ec66ed4d1e848982b610db26f6&scope.project.id=f39c31704e364effbc1f39d76b609302&role.id=8eefae9314f544de97883fa2b10a2046 => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:27.475455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712be543-7777-4fd1-9a62-11d6ce0fb7ca None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:27.529544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a97c660-f447-4f4b-8d2f-b351b986e764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:27.530339 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97c660-f447-4f4b-8d2f-b351b986e764 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:27.530645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97c660-f447-4f4b-8d2f-b351b986e764 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:27.530929 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a97c660-f447-4f4b-8d2f-b351b986e764 None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:27.532133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97c660-f447-4f4b-8d2f-b351b986e764 None admin] RBAC: Authorizing `identity:get_project(project_id=f39c31704e364effbc1f39d76b609302)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:27.533587 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a97c660-f447-4f4b-8d2f-b351b986e764 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:27.535490 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 124/252] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:27 2022] GET /identity/v3/projects/f39c31704e364effbc1f39d76b609302 => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:27.541435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43f81e-4245-4a8d-8338-985f1eae0d68 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:27.667521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a43f81e-4245-4a8d-8338-985f1eae0d68 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:27.703972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4b4d4-fdd2-4a76-a122-1bf86e7dd1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:27.705238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4b4d4-fdd2-4a76-a122-1bf86e7dd1a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:27.705755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4b4d4-fdd2-4a76-a122-1bf86e7dd1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:27.706239 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4b4d4-fdd2-4a76-a122-1bf86e7dd1a8 None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302/users/2be5740ae47448b99ba83deae888b58c/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:27.708432 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4b4d4-fdd2-4a76-a122-1bf86e7dd1a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=f39c31704e364effbc1f39d76b609302, user_id=2be5740ae47448b99ba83deae888b58c, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:27.710763 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4b4d4-fdd2-4a76-a122-1bf86e7dd1a8 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:27.725027 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 129/253] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:47:27 2022] PUT /identity/v3/projects/f39c31704e364effbc1f39d76b609302/users/2be5740ae47448b99ba83deae888b58c/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:47:27.873969 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a397594-afd9-440b-97a3-6641f1250b82 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:27.874328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a397594-afd9-440b-97a3-6641f1250b82 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:27.874617 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a397594-afd9-440b-97a3-6641f1250b82 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:27.876064 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 125/254] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:27.882420 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28992215-a8a3-4959-a2fe-8c3edac9360d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:27.883085 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28992215-a8a3-4959-a2fe-8c3edac9360d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:27.883577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28992215-a8a3-4959-a2fe-8c3edac9360d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:27.903729 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-28992215-a8a3-4959-a2fe-8c3edac9360d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:27.905291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-28992215-a8a3-4959-a2fe-8c3edac9360d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:27.970780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:27.997567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 130/255] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:28.021363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a397594-afd9-440b-97a3-6641f1250b82 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:28.140461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a397594-afd9-440b-97a3-6641f1250b82 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:28.156456 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:28.215313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f618cd6-e2a8-4113-a8f3-d12f63d3d00f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:28.216144 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f618cd6-e2a8-4113-a8f3-d12f63d3d00f None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:28.216434 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f618cd6-e2a8-4113-a8f3-d12f63d3d00f None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:28.216717 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f618cd6-e2a8-4113-a8f3-d12f63d3d00f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:28.231452 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f618cd6-e2a8-4113-a8f3-d12f63d3d00f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:28.239593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-1f618cd6-e2a8-4113-a8f3-d12f63d3d00f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:28.239593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:28.239593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:28.239593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:28.239593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:28.239593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:28.239593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:28.239593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:28.239593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:28.239593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:28.239593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:28.239593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:28.239593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:28.239593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:28.239593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:28.239593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:28.240990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:28.240990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:28.240990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:28.240990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:28.240990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:28.240990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:28.240990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:28.240990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:28.240990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:28.240990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:28.240990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:28.240990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:28.240990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:28.240990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:28.240990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:28.240990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:28.242178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:28.242178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:28.242178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:28.242178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:28.242178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:28.242178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:28.242178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:28.242178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:28.242178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:28.242178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:28.242178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:28.242178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:28.242178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 126/256] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:47:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:28.247224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28992215-a8a3-4959-a2fe-8c3edac9360d None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:28.385691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28992215-a8a3-4959-a2fe-8c3edac9360d None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:28.425157 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf10882-bc9c-4e74-898c-3e394ab8662e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:28.425757 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf10882-bc9c-4e74-898c-3e394ab8662e None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:28.426095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf10882-bc9c-4e74-898c-3e394ab8662e None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:28.426376 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf10882-bc9c-4e74-898c-3e394ab8662e None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:28.427204 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf10882-bc9c-4e74-898c-3e394ab8662e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:28.427519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf10882-bc9c-4e74-898c-3e394ab8662e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:28.429446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf10882-bc9c-4e74-898c-3e394ab8662e None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:28.438244 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 131/257] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:47:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:28.443784 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f618cd6-e2a8-4113-a8f3-d12f63d3d00f None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:28.591669 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f618cd6-e2a8-4113-a8f3-d12f63d3d00f None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:28.622871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544eb57b-6284-467c-8a33-0a04b914fb96 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:28.623185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544eb57b-6284-467c-8a33-0a04b914fb96 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:28.623432 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544eb57b-6284-467c-8a33-0a04b914fb96 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:28.624920 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 132/258] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:47:28.629601 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a1179-5746-4c61-bb4d-4ea723cb435c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:28.630484 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a1179-5746-4c61-bb4d-4ea723cb435c None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:28.630798 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a1179-5746-4c61-bb4d-4ea723cb435c None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:28.631625 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a1179-5746-4c61-bb4d-4ea723cb435c None admin] PATH_INFO: `/v3/users/7e5f84f2dc5942378b26b2db6aa0215f` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:28.635314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a1179-5746-4c61-bb4d-4ea723cb435c None admin] RBAC: Authorizing `identity:get_user(user_id=7e5f84f2dc5942378b26b2db6aa0215f)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:28.636777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946a1179-5746-4c61-bb4d-4ea723cb435c None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:28.638714 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 127/259] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:28 2022] GET /identity/v3/users/7e5f84f2dc5942378b26b2db6aa0215f => generated 283 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:28.643448 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a1179-5746-4c61-bb4d-4ea723cb435c None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:28.644019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31351b7-2cb2-4e2b-aba3-c075c6f89e4d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:28.644315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31351b7-2cb2-4e2b-aba3-c075c6f89e4d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:28.644601 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31351b7-2cb2-4e2b-aba3-c075c6f89e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:28.683712 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-c31351b7-2cb2-4e2b-aba3-c075c6f89e4d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:28.684844 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c31351b7-2cb2-4e2b-aba3-c075c6f89e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:28.745903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a1179-5746-4c61-bb4d-4ea723cb435c None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:28.754952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:28.755577 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:28.824447 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb543d59-8bf5-475b-961e-dfcf90e33dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:28.825120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb543d59-8bf5-475b-961e-dfcf90e33dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:28.825411 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb543d59-8bf5-475b-961e-dfcf90e33dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:28.825695 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb543d59-8bf5-475b-961e-dfcf90e33dc0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:28.828396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 133/260] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:47:28.862717 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31351b7-2cb2-4e2b-aba3-c075c6f89e4d None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:28.958310 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb543d59-8bf5-475b-961e-dfcf90e33dc0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:28.962775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb543d59-8bf5-475b-961e-dfcf90e33dc0 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:29.034966 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:29.057559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31351b7-2cb2-4e2b-aba3-c075c6f89e4d None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:29.153479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 128/261] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 22:47:29.159649 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb543d59-8bf5-475b-961e-dfcf90e33dc0 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:29.168923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b2a5f-8ead-4646-b04b-ee1346a67445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:29.169746 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b2a5f-8ead-4646-b04b-ee1346a67445 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:29.170035 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b2a5f-8ead-4646-b04b-ee1346a67445 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:29.170315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2b2a5f-8ead-4646-b04b-ee1346a67445 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:29.203565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2b2a5f-8ead-4646-b04b-ee1346a67445 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:29.214614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-8d2b2a5f-8ead-4646-b04b-ee1346a67445 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:47:29.214614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:29.214614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:29.214614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:29.214614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:29.214614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:29.214614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:29.214614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:29.214614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:29.214614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:29.214614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:29.214614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:29.214614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:29.214614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:29.214614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:29.214614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:29.215517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:29.215517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:29.215517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:29.215517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:29.215517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:29.215517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:29.215517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:29.215517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:29.215517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:29.215517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:29.215517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:29.215517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:29.215517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:29.215517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:29.215517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:29.215517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:29.216312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:29.216312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:29.216312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:29.216312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:29.216312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:29.216312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:29.216312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:29.216312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:29.216312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:29.216312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:29.216312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:47:29.216312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:29.224260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 134/262] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:47:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 362 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:29.238077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b2a5f-8ead-4646-b04b-ee1346a67445 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:29.347579 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:29.354474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb543d59-8bf5-475b-961e-dfcf90e33dc0 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:29.414859 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16851d9d-f2b4-4227-909c-53984800ca1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:29.415938 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16851d9d-f2b4-4227-909c-53984800ca1f None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:29.416227 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16851d9d-f2b4-4227-909c-53984800ca1f None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:29.416504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16851d9d-f2b4-4227-909c-53984800ca1f None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:29.417703 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16851d9d-f2b4-4227-909c-53984800ca1f None admin] RBAC: Authorizing `identity:get_project(project_id=f39c31704e364effbc1f39d76b609302)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:29.419908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16851d9d-f2b4-4227-909c-53984800ca1f None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:29.428879 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 129/263] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:29 2022] GET /identity/v3/projects/f39c31704e364effbc1f39d76b609302 => generated 302 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:29.435063 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16851d9d-f2b4-4227-909c-53984800ca1f None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:29.450903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:29.524857 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2b2a5f-8ead-4646-b04b-ee1346a67445 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:29.581946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16851d9d-f2b4-4227-909c-53984800ca1f None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:29.617955 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75edc6-c0ed-452a-b2e9-dfc28f2cc187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:29.618158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca282b38-a0d2-4b28-925f-fef0f9e43082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:29.618662 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca282b38-a0d2-4b28-925f-fef0f9e43082 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:29.618955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca282b38-a0d2-4b28-925f-fef0f9e43082 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:29.619235 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca282b38-a0d2-4b28-925f-fef0f9e43082 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:29.619741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75edc6-c0ed-452a-b2e9-dfc28f2cc187 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:29.619997 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca282b38-a0d2-4b28-925f-fef0f9e43082 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:29.620308 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca282b38-a0d2-4b28-925f-fef0f9e43082 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:29.620600 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75edc6-c0ed-452a-b2e9-dfc28f2cc187 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:29.621081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e75edc6-c0ed-452a-b2e9-dfc28f2cc187 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:29.621707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca282b38-a0d2-4b28-925f-fef0f9e43082 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:29.622302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e75edc6-c0ed-452a-b2e9-dfc28f2cc187 None admin] RBAC: Adding query filter params (role.id=bda1c48ca2534bc6b2ca4f3ff479b143, scope.project.id=f39c31704e364effbc1f39d76b609302, user.id=7e5f84f2dc5942378b26b2db6aa0215f) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:29.622833 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e75edc6-c0ed-452a-b2e9-dfc28f2cc187 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:29.624466 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e75edc6-c0ed-452a-b2e9-dfc28f2cc187 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:29.631938 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 130/264] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:47:29 2022] GET /identity/v3/role_assignments?user.id=7e5f84f2dc5942378b26b2db6aa0215f&scope.project.id=f39c31704e364effbc1f39d76b609302&role.id=bda1c48ca2534bc6b2ca4f3ff479b143 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:29.643025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 135/265] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:29.648646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75edc6-c0ed-452a-b2e9-dfc28f2cc187 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:29.738955 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:29.781715 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e75edc6-c0ed-452a-b2e9-dfc28f2cc187 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:29.830349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16654687-b9b1-4c85-aed6-2895d3073a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:29.831674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16654687-b9b1-4c85-aed6-2895d3073a1f None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:29.832158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16654687-b9b1-4c85-aed6-2895d3073a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:29.832627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16654687-b9b1-4c85-aed6-2895d3073a1f None admin] PATH_INFO: `/v3/users/7e5f84f2dc5942378b26b2db6aa0215f` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:29.834725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16654687-b9b1-4c85-aed6-2895d3073a1f None admin] RBAC: Authorizing `identity:get_user(user_id=7e5f84f2dc5942378b26b2db6aa0215f)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:29.836397 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16654687-b9b1-4c85-aed6-2895d3073a1f None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:29.839315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 131/266] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:29 2022] GET /identity/v3/users/7e5f84f2dc5942378b26b2db6aa0215f => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:29.844162 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca282b38-a0d2-4b28-925f-fef0f9e43082 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:29.977458 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:29.985420 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca282b38-a0d2-4b28-925f-fef0f9e43082 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:30.025142 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257575de-5bdc-48f0-a60a-4b8914738272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:30.025810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257575de-5bdc-48f0-a60a-4b8914738272 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:30.026099 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257575de-5bdc-48f0-a60a-4b8914738272 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:30.026379 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257575de-5bdc-48f0-a60a-4b8914738272 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:30.106311 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257575de-5bdc-48f0-a60a-4b8914738272 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:30.107818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257575de-5bdc-48f0-a60a-4b8914738272 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:30.128380 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311335f7-e302-4bac-bbed-325172d25202 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:30.128980 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311335f7-e302-4bac-bbed-325172d25202 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:30.129475 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311335f7-e302-4bac-bbed-325172d25202 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:30.132034 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 132/267] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:30.137260 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6935a81-e870-404d-8774-e6fd42011a81 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:30.137793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6935a81-e870-404d-8774-e6fd42011a81 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:30.138264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6935a81-e870-404d-8774-e6fd42011a81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:30.156352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-c6935a81-e870-404d-8774-e6fd42011a81 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:30.157847 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c6935a81-e870-404d-8774-e6fd42011a81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:30.214620 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:30.219560 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 136/268] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:30.236668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257575de-5bdc-48f0-a60a-4b8914738272 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:30.275795 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 133/269] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:30.284232 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f1e9d-c943-447e-851b-51ece6732ab3 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:30.284755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f1e9d-c943-447e-851b-51ece6732ab3 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:30.285252 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6f1e9d-c943-447e-851b-51ece6732ab3 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:30.291087 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 134/270] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:30.295985 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f0254-59d4-4514-926c-51ad143d8247 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:30.296515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f0254-59d4-4514-926c-51ad143d8247 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:30.296989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5f0254-59d4-4514-926c-51ad143d8247 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:30.324073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-db5f0254-59d4-4514-926c-51ad143d8247 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:30.325562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-db5f0254-59d4-4514-926c-51ad143d8247 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:30.373953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257575de-5bdc-48f0-a60a-4b8914738272 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:30.400095 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:30.408045 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 135/271] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:30.409099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f0254-59d4-4514-926c-51ad143d8247 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:30.412871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21976dec-9895-4769-ae6b-f0af19cac423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:30.415396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21976dec-9895-4769-ae6b-f0af19cac423 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:30.415884 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21976dec-9895-4769-ae6b-f0af19cac423 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:30.416398 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21976dec-9895-4769-ae6b-f0af19cac423 None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:30.417837 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21976dec-9895-4769-ae6b-f0af19cac423 None admin] RBAC: Authorizing `identity:get_project(project_id=f39c31704e364effbc1f39d76b609302)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:30.419567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21976dec-9895-4769-ae6b-f0af19cac423 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:30.421770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 137/272] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:30 2022] GET /identity/v3/projects/f39c31704e364effbc1f39d76b609302 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:30.427680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21976dec-9895-4769-ae6b-f0af19cac423 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:30.489440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:30.523374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5f0254-59d4-4514-926c-51ad143d8247 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:30.531663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21976dec-9895-4769-ae6b-f0af19cac423 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:30.563674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def540ee-2ae2-4dc4-af4d-77b5c75c0856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:30.564746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def540ee-2ae2-4dc4-af4d-77b5c75c0856 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:30.565177 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def540ee-2ae2-4dc4-af4d-77b5c75c0856 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:30.565698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def540ee-2ae2-4dc4-af4d-77b5c75c0856 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:30.571580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a160b81-a24e-425e-ae3a-481811807d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:30.572504 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a160b81-a24e-425e-ae3a-481811807d23 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:30.572933 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a160b81-a24e-425e-ae3a-481811807d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:30.573352 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a160b81-a24e-425e-ae3a-481811807d23 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:30.574444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a160b81-a24e-425e-ae3a-481811807d23 None admin] RBAC: Adding query filter params (role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.project.id=f39c31704e364effbc1f39d76b609302, user.id=7e5f84f2dc5942378b26b2db6aa0215f) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:30.574984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a160b81-a24e-425e-ae3a-481811807d23 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:30.576723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a160b81-a24e-425e-ae3a-481811807d23 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:30.578939 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def540ee-2ae2-4dc4-af4d-77b5c75c0856 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:30.583648 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 138/273] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:47:30 2022] GET /identity/v3/role_assignments?user.id=7e5f84f2dc5942378b26b2db6aa0215f&scope.project.id=f39c31704e364effbc1f39d76b609302&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:30.585182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a160b81-a24e-425e-ae3a-481811807d23 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:30.588073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-def540ee-2ae2-4dc4-af4d-77b5c75c0856 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:30.588073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:30.588073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:30.588073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:30.588073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:30.588073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:30.588073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:30.588073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:30.588073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:30.588073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:30.588073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:30.588073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:30.588073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:30.588073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:30.588073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:30.588073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:30.588944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:30.588944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:30.588944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:30.588944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:30.588944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:30.588944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:30.588944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:30.588944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:30.588944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:30.588944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:30.588944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:30.588944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:30.588944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:30.588944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:30.588944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:30.588944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:30.589727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:30.589727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:30.589727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:30.589727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:30.589727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:30.589727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:30.589727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:30.589727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:30.589727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:30.589727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:30.589727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:30.589727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:30.589727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 136/274] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:30.603865 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def540ee-2ae2-4dc4-af4d-77b5c75c0856 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:30.701154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:30.707890 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a160b81-a24e-425e-ae3a-481811807d23 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:30.744548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:30.755235 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3f25c-e53a-4528-ad30-a7148efbf366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:30.756946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3f25c-e53a-4528-ad30-a7148efbf366 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:30.757430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3f25c-e53a-4528-ad30-a7148efbf366 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:30.757556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def540ee-2ae2-4dc4-af4d-77b5c75c0856 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:30.758032 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3f25c-e53a-4528-ad30-a7148efbf366 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:30.770211 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3f25c-e53a-4528-ad30-a7148efbf366 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:30.779189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-dae3f25c-e53a-4528-ad30-a7148efbf366 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 22:47:30.779189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:30.779189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:30.779189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:30.779189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:30.779189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:30.779189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:30.779189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:30.779189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:30.779189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:30.779189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:30.779189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:30.779189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:30.779189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:30.779189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:30.779189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:30.780126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:30.780126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:30.780126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:30.780126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:30.780126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:30.780126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:30.780126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:30.780126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:30.780126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:30.780126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:30.780126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:30.780126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:30.780126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:30.780126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:30.780126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:30.780126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:30.780894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:30.780894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:30.780894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:30.780894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:30.780894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:30.780894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:30.780894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:30.780894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:30.780894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:30.780894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:30.780894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 22:47:30.780894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:30.780894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 139/275] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:30.784538 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd05fa5-6d0a-4f85-9d0e-5a9ef3d5db93 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:30.786727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd05fa5-6d0a-4f85-9d0e-5a9ef3d5db93 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:30.787181 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd05fa5-6d0a-4f85-9d0e-5a9ef3d5db93 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:30.789261 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 140/276] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:30.790185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd05fa5-6d0a-4f85-9d0e-5a9ef3d5db93 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:30.801262 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8fc01-a3dc-4849-a488-e271082f4983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:30.801971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8fc01-a3dc-4849-a488-e271082f4983 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:30.802261 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8fc01-a3dc-4849-a488-e271082f4983 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:30.802540 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8fc01-a3dc-4849-a488-e271082f4983 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:30.804775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8fc01-a3dc-4849-a488-e271082f4983 None admin] RBAC: Adding query filter params (name=member) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:30.804775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8fc01-a3dc-4849-a488-e271082f4983 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:30.806541 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8fc01-a3dc-4849-a488-e271082f4983 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:30.815497 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 137/277] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:30.816873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f85064-b2ea-47c7-b8b1-e5dd3bb8ace9 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:30.817165 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f85064-b2ea-47c7-b8b1-e5dd3bb8ace9 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:30.817447 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f85064-b2ea-47c7-b8b1-e5dd3bb8ace9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:30.838303 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-95f85064-b2ea-47c7-b8b1-e5dd3bb8ace9 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:30.839644 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-95f85064-b2ea-47c7-b8b1-e5dd3bb8ace9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:30.921109 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd05fa5-6d0a-4f85-9d0e-5a9ef3d5db93 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:30.931094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 138/278] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:30.931094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f85064-b2ea-47c7-b8b1-e5dd3bb8ace9 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:30.961744 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f776fe6-4d4b-4836-82e3-8e61b3336a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:30.962412 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f776fe6-4d4b-4836-82e3-8e61b3336a6d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:30.962718 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f776fe6-4d4b-4836-82e3-8e61b3336a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:30.963025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f776fe6-4d4b-4836-82e3-8e61b3336a6d None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:30.963901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f776fe6-4d4b-4836-82e3-8e61b3336a6d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:30.964214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f776fe6-4d4b-4836-82e3-8e61b3336a6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:30.965703 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f776fe6-4d4b-4836-82e3-8e61b3336a6d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:30.976300 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 141/279] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:30.981305 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f776fe6-4d4b-4836-82e3-8e61b3336a6d None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:30.984122 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:31.035863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:31.043283 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f85064-b2ea-47c7-b8b1-e5dd3bb8ace9 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:31.083493 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6b92e-2e6e-440f-bbbe-1c4f1c57ed17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:31.084983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6b92e-2e6e-440f-bbbe-1c4f1c57ed17 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:31.085263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6b92e-2e6e-440f-bbbe-1c4f1c57ed17 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:31.085537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b6b92e-2e6e-440f-bbbe-1c4f1c57ed17 None admin] PATH_INFO: `/v3/users/6bd7eecb16ed429f8a651a245706a341` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:31.088213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f776fe6-4d4b-4836-82e3-8e61b3336a6d None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:31.098405 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6b92e-2e6e-440f-bbbe-1c4f1c57ed17 None admin] RBAC: Authorizing `identity:get_user(user_id=6bd7eecb16ed429f8a651a245706a341)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:31.099940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b6b92e-2e6e-440f-bbbe-1c4f1c57ed17 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:31.101595 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 139/280] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:30 2022] GET /identity/v3/users/6bd7eecb16ed429f8a651a245706a341 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:31.102426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6b92e-2e6e-440f-bbbe-1c4f1c57ed17 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:31.122354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b6c4b9-f556-4c0a-88ba-adff83701aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:31.123177 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6c4b9-f556-4c0a-88ba-adff83701aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:31.123463 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6c4b9-f556-4c0a-88ba-adff83701aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:31.123740 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b6c4b9-f556-4c0a-88ba-adff83701aa3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:31.124881 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6c4b9-f556-4c0a-88ba-adff83701aa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:31.126342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b6c4b9-f556-4c0a-88ba-adff83701aa3 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:31.127981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 142/281] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:47:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:31.129133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ab6ee-3215-42f4-8fe2-0e555a40d72c None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:31.129416 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ab6ee-3215-42f4-8fe2-0e555a40d72c None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:31.129690 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662ab6ee-3215-42f4-8fe2-0e555a40d72c None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:31.130926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 143/282] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:31.132135 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ab6ee-3215-42f4-8fe2-0e555a40d72c None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:31.207332 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b6b92e-2e6e-440f-bbbe-1c4f1c57ed17 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:31.217095 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:31.232315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662ab6ee-3215-42f4-8fe2-0e555a40d72c None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:31.241825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac79ba-a3d3-4901-841a-9fc4736f13d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:31.242729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac79ba-a3d3-4901-841a-9fc4736f13d4 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:31.243012 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac79ba-a3d3-4901-841a-9fc4736f13d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:31.243288 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aac79ba-a3d3-4901-841a-9fc4736f13d4 None admin] PATH_INFO: `/v3/users/911af5ec66ed4d1e848982b610db26f6` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:31.244465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac79ba-a3d3-4901-841a-9fc4736f13d4 None admin] RBAC: Authorizing `identity:get_user(user_id=911af5ec66ed4d1e848982b610db26f6)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:31.246002 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aac79ba-a3d3-4901-841a-9fc4736f13d4 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:31.247560 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 140/283] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:31 2022] GET /identity/v3/users/911af5ec66ed4d1e848982b610db26f6 => generated 297 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:31.252121 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac79ba-a3d3-4901-841a-9fc4736f13d4 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:31.252195 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:31.268251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7821c83a-5bb6-46ab-ae1d-ea2ece02cc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:31.268912 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821c83a-5bb6-46ab-ae1d-ea2ece02cc2a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:31.269196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821c83a-5bb6-46ab-ae1d-ea2ece02cc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:31.269474 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7821c83a-5bb6-46ab-ae1d-ea2ece02cc2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:31.270295 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821c83a-5bb6-46ab-ae1d-ea2ece02cc2a None admin] RBAC: Adding query filter params (role.id=bff4f694fb57451e9a0783dc014187f2, user.id=6bd7eecb16ed429f8a651a245706a341, scope.system=all) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:31.270633 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821c83a-5bb6-46ab-ae1d-ea2ece02cc2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:31.272617 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7821c83a-5bb6-46ab-ae1d-ea2ece02cc2a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:31.290123 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 144/284] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jun 16 22:47:31 2022] GET /identity/v3/role_assignments?user.id=6bd7eecb16ed429f8a651a245706a341&scope.system=all&role.id=bff4f694fb57451e9a0783dc014187f2 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:31.290123 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea6522-eb7a-4849-b411-c19f50e63b31 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:31.290123 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea6522-eb7a-4849-b411-c19f50e63b31 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:31.290123 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ea6522-eb7a-4849-b411-c19f50e63b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:31.304274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-74ea6522-eb7a-4849-b411-c19f50e63b31 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:31.305641 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-74ea6522-eb7a-4849-b411-c19f50e63b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:31.365634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aac79ba-a3d3-4901-841a-9fc4736f13d4 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:31.380594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 145/285] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:31.383820 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea6522-eb7a-4849-b411-c19f50e63b31 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:31.406907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8a399-6ddb-43c7-9f7d-536e8afdb961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:31.407596 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8a399-6ddb-43c7-9f7d-536e8afdb961 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:31.408214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8a399-6ddb-43c7-9f7d-536e8afdb961 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:31.408501 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8a399-6ddb-43c7-9f7d-536e8afdb961 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:31.409235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8a399-6ddb-43c7-9f7d-536e8afdb961 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:31.410731 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8a399-6ddb-43c7-9f7d-536e8afdb961 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:31.442616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 141/286] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 22:47:31 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:47:31.442616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8a399-6ddb-43c7-9f7d-536e8afdb961 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:31.482691 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:31.529967 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ea6522-eb7a-4849-b411-c19f50e63b31 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:31.542772 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:31.566503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8a399-6ddb-43c7-9f7d-536e8afdb961 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:31.570677 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f456a-a6fb-4fe6-83fd-f3a784c7f134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:31.571778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f456a-a6fb-4fe6-83fd-f3a784c7f134 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:31.572249 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f456a-a6fb-4fe6-83fd-f3a784c7f134 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:31.572709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5f456a-a6fb-4fe6-83fd-f3a784c7f134 None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:31.574196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f456a-a6fb-4fe6-83fd-f3a784c7f134 None admin] RBAC: Authorizing `identity:get_project(project_id=f39c31704e364effbc1f39d76b609302)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:31.575941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5f456a-a6fb-4fe6-83fd-f3a784c7f134 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:31.578256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 146/287] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:31 2022] GET /identity/v3/projects/f39c31704e364effbc1f39d76b609302 => generated 302 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:31.586264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f456a-a6fb-4fe6-83fd-f3a784c7f134 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:31.606081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3bad94-d766-4600-bfff-2d054f60dd16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:31.606915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bad94-d766-4600-bfff-2d054f60dd16 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:31.607199 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bad94-d766-4600-bfff-2d054f60dd16 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:31.608948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3bad94-d766-4600-bfff-2d054f60dd16 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:31.623650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3bad94-d766-4600-bfff-2d054f60dd16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:31.631944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-0c3bad94-d766-4600-bfff-2d054f60dd16 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:31.631944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:31.631944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:31.631944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:31.631944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:31.631944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:31.631944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:31.631944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:31.631944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:31.631944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:31.631944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:31.631944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:31.631944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:31.631944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:31.631944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:31.631944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:31.632858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:31.632858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:31.632858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:31.632858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:31.632858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:31.632858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:31.632858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:31.632858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:31.632858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:31.632858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:31.632858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:31.632858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:31.632858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:31.632858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:31.632858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:31.632858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:31.633841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:31.633841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:31.633841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:31.633841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:31.633841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:31.633841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:31.633841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:31.633841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:31.633841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:31.633841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:31.633841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:31.633841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:31.633841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 142/288] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:31.638745 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3bad94-d766-4600-bfff-2d054f60dd16 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:31.720380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5f456a-a6fb-4fe6-83fd-f3a784c7f134 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:31.745574 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:31.761630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3bad94-d766-4600-bfff-2d054f60dd16 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:31.773608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945824ff-b314-4836-9ec4-b67eceebc9d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:31.774517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945824ff-b314-4836-9ec4-b67eceebc9d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:31.774836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945824ff-b314-4836-9ec4-b67eceebc9d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:31.775116 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945824ff-b314-4836-9ec4-b67eceebc9d2 None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302/users/911af5ec66ed4d1e848982b610db26f6/roles/8eefae9314f544de97883fa2b10a2046` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:31.777073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945824ff-b314-4836-9ec4-b67eceebc9d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f39c31704e364effbc1f39d76b609302, user_id=911af5ec66ed4d1e848982b610db26f6, role_id=8eefae9314f544de97883fa2b10a2046)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:31.779355 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945824ff-b314-4836-9ec4-b67eceebc9d2 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:31.795388 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 147/289] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:47:31 2022] PUT /identity/v3/projects/f39c31704e364effbc1f39d76b609302/users/911af5ec66ed4d1e848982b610db26f6/roles/8eefae9314f544de97883fa2b10a2046 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:47:31.807877 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413c716-04ed-4711-826c-e0272b428162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:31.808608 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413c716-04ed-4711-826c-e0272b428162 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:31.808897 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413c716-04ed-4711-826c-e0272b428162 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:31.809179 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0413c716-04ed-4711-826c-e0272b428162 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:31.809980 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413c716-04ed-4711-826c-e0272b428162 None admin] RBAC: Adding query filter params (name=member) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:31.810294 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413c716-04ed-4711-826c-e0272b428162 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:31.811767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0413c716-04ed-4711-826c-e0272b428162 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:31.822317 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 143/290] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:31.828499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945824ff-b314-4836-9ec4-b67eceebc9d2 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:31.873407 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:31.947113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945824ff-b314-4836-9ec4-b67eceebc9d2 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:31.987609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504c133-b4f0-42e0-8e2a-a7493929174b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:31.988521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504c133-b4f0-42e0-8e2a-a7493929174b None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:31.988805 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504c133-b4f0-42e0-8e2a-a7493929174b None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:31.989084 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0504c133-b4f0-42e0-8e2a-a7493929174b None admin] PATH_INFO: `/v3/users/2be5740ae47448b99ba83deae888b58c` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:31.990319 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504c133-b4f0-42e0-8e2a-a7493929174b None admin] RBAC: Authorizing `identity:get_user(user_id=2be5740ae47448b99ba83deae888b58c)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:31.992530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0504c133-b4f0-42e0-8e2a-a7493929174b None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:31.994479 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 148/291] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:31 2022] GET /identity/v3/users/2be5740ae47448b99ba83deae888b58c => generated 297 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:32.000085 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413c716-04ed-4711-826c-e0272b428162 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:32.111677 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0413c716-04ed-4711-826c-e0272b428162 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:32.150876 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad1bfd-86e9-4f33-97ec-da251daaf24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:32.151537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad1bfd-86e9-4f33-97ec-da251daaf24e None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:32.151820 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad1bfd-86e9-4f33-97ec-da251daaf24e None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:32.152098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ad1bfd-86e9-4f33-97ec-da251daaf24e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:32.177271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:32.223343 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad1bfd-86e9-4f33-97ec-da251daaf24e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:32.224802 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ad1bfd-86e9-4f33-97ec-da251daaf24e None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:32.326195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 144/292] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:32.335691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504c133-b4f0-42e0-8e2a-a7493929174b None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:32.481366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0504c133-b4f0-42e0-8e2a-a7493929174b None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:32.488965 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:32.518662 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a9eceb-f350-4e91-890c-8f4b1b60bf56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:32.519542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9eceb-f350-4e91-890c-8f4b1b60bf56 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:32.519827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9eceb-f350-4e91-890c-8f4b1b60bf56 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:32.520105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a9eceb-f350-4e91-890c-8f4b1b60bf56 None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:32.521367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9eceb-f350-4e91-890c-8f4b1b60bf56 None admin] RBAC: Authorizing `identity:get_project(project_id=f39c31704e364effbc1f39d76b609302)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:32.522898 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a9eceb-f350-4e91-890c-8f4b1b60bf56 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:32.524783 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 149/293] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:32 2022] GET /identity/v3/projects/f39c31704e364effbc1f39d76b609302 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:32.530788 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad1bfd-86e9-4f33-97ec-da251daaf24e None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:32.587327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:32.634296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ad1bfd-86e9-4f33-97ec-da251daaf24e None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:32.668610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f73a82c-b0ff-4df0-aa83-2a1ec6c14b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:32.669270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73a82c-b0ff-4df0-aa83-2a1ec6c14b8f None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:32.669557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73a82c-b0ff-4df0-aa83-2a1ec6c14b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:32.669832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f73a82c-b0ff-4df0-aa83-2a1ec6c14b8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:32.670695 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73a82c-b0ff-4df0-aa83-2a1ec6c14b8f None admin] RBAC: Adding query filter params (role.id=bff4f694fb57451e9a0783dc014187f2, scope.project.id=f39c31704e364effbc1f39d76b609302, user.id=2be5740ae47448b99ba83deae888b58c) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:32.671012 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73a82c-b0ff-4df0-aa83-2a1ec6c14b8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:32.672471 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f73a82c-b0ff-4df0-aa83-2a1ec6c14b8f None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:32.679596 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 145/294] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:47:32 2022] GET /identity/v3/role_assignments?user.id=2be5740ae47448b99ba83deae888b58c&scope.project.id=f39c31704e364effbc1f39d76b609302&role.id=bff4f694fb57451e9a0783dc014187f2 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:33.615383 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338ceea-a8c8-4dd1-a84a-db762cac533f None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:33.616464 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338ceea-a8c8-4dd1-a84a-db762cac533f None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:33.616464 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c338ceea-a8c8-4dd1-a84a-db762cac533f None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:33.618120 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 150/295] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:33.623701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e3b9a-0f2b-4738-ad7d-4e2426581fe8 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:33.624086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e3b9a-0f2b-4738-ad7d-4e2426581fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:33.624263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e3b9a-0f2b-4738-ad7d-4e2426581fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:33.644882 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-9b5e3b9a-0f2b-4738-ad7d-4e2426581fe8 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:33.646356 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-9b5e3b9a-0f2b-4738-ad7d-4e2426581fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:33.715106 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 146/296] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:33.736909 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338ceea-a8c8-4dd1-a84a-db762cac533f None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:33.848468 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c338ceea-a8c8-4dd1-a84a-db762cac533f None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:33.883929 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05347cb-40ad-452c-9993-2aadb0f29baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:33.884650 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05347cb-40ad-452c-9993-2aadb0f29baf None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:33.884935 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05347cb-40ad-452c-9993-2aadb0f29baf None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:33.885209 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05347cb-40ad-452c-9993-2aadb0f29baf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:33.895955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05347cb-40ad-452c-9993-2aadb0f29baf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:33.903925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-b05347cb-40ad-452c-9993-2aadb0f29baf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:33.903925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:33.903925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:33.903925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:33.903925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:33.903925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:33.903925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:33.903925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:33.903925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:33.903925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:33.903925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:33.903925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:33.903925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:33.903925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:33.903925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:33.903925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:33.904776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:33.904776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:33.904776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:33.904776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:33.904776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:33.904776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:33.904776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:33.904776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:33.904776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:33.904776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:33.904776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:33.904776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:33.904776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:33.904776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:33.904776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:33.904776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:33.905540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:33.905540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:33.905540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:33.905540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:33.905540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:33.905540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:33.905540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:33.905540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:33.905540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:33.905540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:33.905540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:33.905540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:33.905540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 151/297] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:33.911248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e3b9a-0f2b-4738-ad7d-4e2426581fe8 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:33.963759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:34.016850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e3b9a-0f2b-4738-ad7d-4e2426581fe8 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:34.056485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413ca84-e621-44d3-851d-888066340966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:34.057678 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413ca84-e621-44d3-851d-888066340966 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:34.058196 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413ca84-e621-44d3-851d-888066340966 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:34.058583 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413ca84-e621-44d3-851d-888066340966 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:34.059418 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413ca84-e621-44d3-851d-888066340966 None admin] RBAC: Adding query filter params (name=member) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:34.059739 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413ca84-e621-44d3-851d-888066340966 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:34.061179 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413ca84-e621-44d3-851d-888066340966 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:34.086550 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 147/298] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:34.087951 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05347cb-40ad-452c-9993-2aadb0f29baf None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:34.107567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217cd14-b97a-43ec-a934-f96315dfca69 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:34.108181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217cd14-b97a-43ec-a934-f96315dfca69 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:34.108662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9217cd14-b97a-43ec-a934-f96315dfca69 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:34.110494 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 148/299] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:47:34.119951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e2a1d-f08a-4cfb-8c92-92ad74c1a7c6 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:34.120238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e2a1d-f08a-4cfb-8c92-92ad74c1a7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:34.120528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0e2a1d-f08a-4cfb-8c92-92ad74c1a7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:34.133888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:34.143566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-ab0e2a1d-f08a-4cfb-8c92-92ad74c1a7c6 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:34.145050 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ab0e2a1d-f08a-4cfb-8c92-92ad74c1a7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:34.215852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05347cb-40ad-452c-9993-2aadb0f29baf None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:34.243200 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 149/300] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:47:34.252002 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c3bcc-924b-449f-8680-922f5a22f74b None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:34.252293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c3bcc-924b-449f-8680-922f5a22f74b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:34.252579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c3bcc-924b-449f-8680-922f5a22f74b None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:34.253875 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 150/301] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:34.259337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b86eb5-4732-4833-8f69-b076035cb8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:34.259606 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99d686-73c2-47f4-86ef-2c91a0128368 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:34.260087 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99d686-73c2-47f4-86ef-2c91a0128368 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:34.260256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b86eb5-4732-4833-8f69-b076035cb8e8 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:34.260494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b86eb5-4732-4833-8f69-b076035cb8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:34.260770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b86eb5-4732-4833-8f69-b076035cb8e8 None admin] PATH_INFO: `/v3/users/6bd7eecb16ed429f8a651a245706a341` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:34.261135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a99d686-73c2-47f4-86ef-2c91a0128368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:34.261951 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b86eb5-4732-4833-8f69-b076035cb8e8 None admin] RBAC: Authorizing `identity:get_user(user_id=6bd7eecb16ed429f8a651a245706a341)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:34.263580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b86eb5-4732-4833-8f69-b076035cb8e8 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:34.265482 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 152/302] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:34 2022] GET /identity/v3/users/6bd7eecb16ed429f8a651a245706a341 => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:34.272354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b86eb5-4732-4833-8f69-b076035cb8e8 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:34.280597 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-7a99d686-73c2-47f4-86ef-2c91a0128368 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:34.281791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7a99d686-73c2-47f4-86ef-2c91a0128368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:34.324797 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:34.363366 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 151/303] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:47:34.371703 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99d686-73c2-47f4-86ef-2c91a0128368 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:34.386628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b86eb5-4732-4833-8f69-b076035cb8e8 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:34.388277 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:34.424023 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5d67e-d881-4049-8688-f8a63e252245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:34.424821 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5d67e-d881-4049-8688-f8a63e252245 None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:34.425107 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5d67e-d881-4049-8688-f8a63e252245 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:34.425388 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac5d67e-d881-4049-8688-f8a63e252245 None admin] PATH_INFO: `/v3/system/users/6bd7eecb16ed429f8a651a245706a341/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:34.426913 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5d67e-d881-4049-8688-f8a63e252245 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=6bd7eecb16ed429f8a651a245706a341, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:34.428823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac5d67e-d881-4049-8688-f8a63e252245 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:34.437736 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 153/304] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jun 16 22:47:34 2022] PUT /identity/v3/system/users/6bd7eecb16ed429f8a651a245706a341/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 22:47:34.447943 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5d67e-d881-4049-8688-f8a63e252245 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:34.487314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a99d686-73c2-47f4-86ef-2c91a0128368 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:34.525673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a343ede-4727-4c37-a894-e8cca9286478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:34.526494 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a343ede-4727-4c37-a894-e8cca9286478 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:34.526810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a343ede-4727-4c37-a894-e8cca9286478 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:34.527086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a343ede-4727-4c37-a894-e8cca9286478 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:34.540717 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a343ede-4727-4c37-a894-e8cca9286478 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:34.548582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-4a343ede-4727-4c37-a894-e8cca9286478 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 22:47:34.548582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:34.548582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:34.548582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:34.548582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:34.548582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:34.548582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:34.548582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:34.548582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:34.548582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:34.548582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:34.548582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:34.548582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:34.548582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:34.548582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:34.548582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:34.549566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:34.549566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:34.549566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:34.549566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:34.549566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:34.549566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:34.549566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:34.549566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:34.549566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:34.549566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:34.549566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:34.549566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:34.549566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:34.549566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:34.549566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:34.549566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:34.550444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:34.550444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:34.550444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:34.550444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:34.550444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:34.550444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:34.550444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:34.550444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:34.550444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:34.550444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:34.550444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 22:47:34.550444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:34.550444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 152/305] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:34.551648 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161c550-cf7e-4eed-88ab-1d291418d9a4 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:34.551811 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161c550-cf7e-4eed-88ab-1d291418d9a4 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:34.552061 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0161c550-cf7e-4eed-88ab-1d291418d9a4 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:34.553670 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 153/306] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:34.558415 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161c550-cf7e-4eed-88ab-1d291418d9a4 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:34.569593 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac5d67e-d881-4049-8688-f8a63e252245 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:34.592238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:34.611130 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8adbe89d-fa0d-4859-a60a-453724eac7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:34.612491 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbe89d-fa0d-4859-a60a-453724eac7a2 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:34.612776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbe89d-fa0d-4859-a60a-453724eac7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:34.613052 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8adbe89d-fa0d-4859-a60a-453724eac7a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:34.614369 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adbe89d-fa0d-4859-a60a-453724eac7a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:34.616303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8adbe89d-fa0d-4859-a60a-453724eac7a2 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:34.618404 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 154/307] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:47:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:34.619145 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5cf91-1c25-4439-95a0-34482c4fd9bf None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:34.619433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5cf91-1c25-4439-95a0-34482c4fd9bf None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:34.619710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b5cf91-1c25-4439-95a0-34482c4fd9bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:34.637690 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-f2b5cf91-1c25-4439-95a0-34482c4fd9bf None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:34.639194 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f2b5cf91-1c25-4439-95a0-34482c4fd9bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:34.672216 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:34.673739 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0161c550-cf7e-4eed-88ab-1d291418d9a4 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:34.711178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055d7b3-2c7d-4cd5-9c70-865ab0a39f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:34.711764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055d7b3-2c7d-4cd5-9c70-865ab0a39f63 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:34.712051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055d7b3-2c7d-4cd5-9c70-865ab0a39f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:34.712326 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8055d7b3-2c7d-4cd5-9c70-865ab0a39f63 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:34.713066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055d7b3-2c7d-4cd5-9c70-865ab0a39f63 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:34.713373 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055d7b3-2c7d-4cd5-9c70-865ab0a39f63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:34.714759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8055d7b3-2c7d-4cd5-9c70-865ab0a39f63 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:34.721543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 155/308] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:34.722350 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5cf91-1c25-4439-95a0-34482c4fd9bf None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:34.733365 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 154/309] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:34.738359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055d7b3-2c7d-4cd5-9c70-865ab0a39f63 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:34.835759 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b5cf91-1c25-4439-95a0-34482c4fd9bf None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:34.836645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:34.847916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8055d7b3-2c7d-4cd5-9c70-865ab0a39f63 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:34.876654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58c14d-7dd0-4cbe-bf7a-14881049571f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:34.877332 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58c14d-7dd0-4cbe-bf7a-14881049571f None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:34.877630 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58c14d-7dd0-4cbe-bf7a-14881049571f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:34.877911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d58c14d-7dd0-4cbe-bf7a-14881049571f None admin] PATH_INFO: `/v3/groups` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:34.878619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58c14d-7dd0-4cbe-bf7a-14881049571f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:34.880170 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d58c14d-7dd0-4cbe-bf7a-14881049571f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:34.884184 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6dead-e00e-4582-87c1-c7cec6fa26e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:34.885169 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6dead-e00e-4582-87c1-c7cec6fa26e8 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:34.885660 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6dead-e00e-4582-87c1-c7cec6fa26e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:34.885895 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6dead-e00e-4582-87c1-c7cec6fa26e8 None admin] PATH_INFO: `/v3/users/90b4403d4c9e4966acbd173bd8e8255a` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:34.897708 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 156/310] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jun 16 22:47:34 2022] POST /identity/v3/groups => generated 229 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:47:34.902236 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58c14d-7dd0-4cbe-bf7a-14881049571f None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:34.903682 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6dead-e00e-4582-87c1-c7cec6fa26e8 None admin] RBAC: Authorizing `identity:get_user(user_id=90b4403d4c9e4966acbd173bd8e8255a)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:34.905445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6dead-e00e-4582-87c1-c7cec6fa26e8 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:34.907430 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 155/311] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:34 2022] GET /identity/v3/users/90b4403d4c9e4966acbd173bd8e8255a => generated 293 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:34.913504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6dead-e00e-4582-87c1-c7cec6fa26e8 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:34.942377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:35.051603 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6dead-e00e-4582-87c1-c7cec6fa26e8 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:35.054229 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d58c14d-7dd0-4cbe-bf7a-14881049571f None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:35.098561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd354de-ae19-48b3-b11b-62c626f282df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:35.099535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd354de-ae19-48b3-b11b-62c626f282df None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:35.099867 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd354de-ae19-48b3-b11b-62c626f282df None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:35.100167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd354de-ae19-48b3-b11b-62c626f282df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:35.101016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd354de-ae19-48b3-b11b-62c626f282df None admin] RBAC: Adding query filter params (role.id=8eefae9314f544de97883fa2b10a2046, user.id=90b4403d4c9e4966acbd173bd8e8255a, scope.system=all) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:35.101339 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd354de-ae19-48b3-b11b-62c626f282df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:35.102331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed1f3f-58af-4ab7-bb01-5595786ca1da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:35.103273 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd354de-ae19-48b3-b11b-62c626f282df None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:35.103389 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed1f3f-58af-4ab7-bb01-5595786ca1da None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:35.103461 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed1f3f-58af-4ab7-bb01-5595786ca1da None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:35.103719 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed1f3f-58af-4ab7-bb01-5595786ca1da None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:35.109831 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 156/312] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jun 16 22:47:34 2022] GET /identity/v3/role_assignments?user.id=90b4403d4c9e4966acbd173bd8e8255a&scope.system=all&role.id=8eefae9314f544de97883fa2b10a2046 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:35.117447 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed1f3f-58af-4ab7-bb01-5595786ca1da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:35.125367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-ceed1f3f-58af-4ab7-bb01-5595786ca1da None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 22:47:35.125367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:35.125367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:35.125367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:35.125367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:35.125367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:35.125367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:35.125367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:35.125367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:35.125367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:35.125367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:35.125367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:35.125367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:35.125367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:35.125367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:35.125367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:35.126328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:35.126328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:35.126328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:35.126328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:35.126328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:35.126328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:35.126328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:35.126328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:35.126328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:35.126328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:35.126328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:35.126328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:35.126328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:35.126328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:35.126328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:35.126328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:35.127150 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:35.127150 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:35.127150 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:35.127150 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:35.127150 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:35.127150 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:35.127150 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:35.127150 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:35.127150 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:35.127150 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:35.127150 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 22:47:35.127150 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:35.127150 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 157/313] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:35.132874 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd354de-ae19-48b3-b11b-62c626f282df None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:35.154996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:35.246993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd354de-ae19-48b3-b11b-62c626f282df None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:35.284409 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa4580-8c9d-422f-81ab-0cd9fc52dcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:35.285088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa4580-8c9d-422f-81ab-0cd9fc52dcc2 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:35.285375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa4580-8c9d-422f-81ab-0cd9fc52dcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:35.285655 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaa4580-8c9d-422f-81ab-0cd9fc52dcc2 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:35.286414 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa4580-8c9d-422f-81ab-0cd9fc52dcc2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:35.286753 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa4580-8c9d-422f-81ab-0cd9fc52dcc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:35.288682 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eaa4580-8c9d-422f-81ab-0cd9fc52dcc2 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:35.297382 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 157/314] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:35.302954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed1f3f-58af-4ab7-bb01-5595786ca1da None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:35.395236 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:35.413392 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed1f3f-58af-4ab7-bb01-5595786ca1da None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:35.448618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db9fc3-e5c5-4686-b7e4-128af0ca1449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:35.449918 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db9fc3-e5c5-4686-b7e4-128af0ca1449 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:35.450421 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db9fc3-e5c5-4686-b7e4-128af0ca1449 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:35.450901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48db9fc3-e5c5-4686-b7e4-128af0ca1449 None admin] PATH_INFO: `/v3/users/911af5ec66ed4d1e848982b610db26f6` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:35.452274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db9fc3-e5c5-4686-b7e4-128af0ca1449 None admin] RBAC: Authorizing `identity:get_user(user_id=911af5ec66ed4d1e848982b610db26f6)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:35.453928 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48db9fc3-e5c5-4686-b7e4-128af0ca1449 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:35.455786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 158/315] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:35 2022] GET /identity/v3/users/911af5ec66ed4d1e848982b610db26f6 => generated 297 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:35.461970 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa4580-8c9d-422f-81ab-0cd9fc52dcc2 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:35.543176 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:35.567333 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eaa4580-8c9d-422f-81ab-0cd9fc52dcc2 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:35.605673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55857da4-d09c-41cd-bdd8-593523c9fa85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:35.606327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55857da4-d09c-41cd-bdd8-593523c9fa85 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:35.606627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55857da4-d09c-41cd-bdd8-593523c9fa85 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:35.606912 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55857da4-d09c-41cd-bdd8-593523c9fa85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:35.684448 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55857da4-d09c-41cd-bdd8-593523c9fa85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:35.685975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55857da4-d09c-41cd-bdd8-593523c9fa85 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:35.769055 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:35.798476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 158/316] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:35.804662 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db9fc3-e5c5-4686-b7e4-128af0ca1449 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:35.909570 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48db9fc3-e5c5-4686-b7e4-128af0ca1449 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:35.943509 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2765d62-38a5-4df4-a43a-b97f6f8dc455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:35.944401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2765d62-38a5-4df4-a43a-b97f6f8dc455 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:35.944682 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2765d62-38a5-4df4-a43a-b97f6f8dc455 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:35.944958 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2765d62-38a5-4df4-a43a-b97f6f8dc455 None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:35.946120 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2765d62-38a5-4df4-a43a-b97f6f8dc455 None admin] RBAC: Authorizing `identity:get_project(project_id=f39c31704e364effbc1f39d76b609302)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:35.947688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2765d62-38a5-4df4-a43a-b97f6f8dc455 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:35.949924 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 159/317] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:35 2022] GET /identity/v3/projects/f39c31704e364effbc1f39d76b609302 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:35.955790 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55857da4-d09c-41cd-bdd8-593523c9fa85 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:36.059655 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55857da4-d09c-41cd-bdd8-593523c9fa85 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:36.096384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f697976d-43ed-47c3-85e2-bf822a85543e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:36.097074 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697976d-43ed-47c3-85e2-bf822a85543e None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:36.097357 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697976d-43ed-47c3-85e2-bf822a85543e None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:36.097636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697976d-43ed-47c3-85e2-bf822a85543e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:36.098496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f697976d-43ed-47c3-85e2-bf822a85543e None admin] RBAC: Adding query filter params (role.id=8eefae9314f544de97883fa2b10a2046, scope.project.id=f39c31704e364effbc1f39d76b609302, user.id=911af5ec66ed4d1e848982b610db26f6) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:36.098842 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f697976d-43ed-47c3-85e2-bf822a85543e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:36.100837 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f697976d-43ed-47c3-85e2-bf822a85543e None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:36.107747 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 159/318] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:47:35 2022] GET /identity/v3/role_assignments?user.id=911af5ec66ed4d1e848982b610db26f6&scope.project.id=f39c31704e364effbc1f39d76b609302&role.id=8eefae9314f544de97883fa2b10a2046 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:36.707538 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4caea9-2571-4a76-8769-367db4f652f8 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:36.707903 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4caea9-2571-4a76-8769-367db4f652f8 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:36.708105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4caea9-2571-4a76-8769-367db4f652f8 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:36.710047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 160/319] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:36.715044 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d89f2a-7d70-4cec-b878-68df2d85693a None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:36.715490 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d89f2a-7d70-4cec-b878-68df2d85693a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:36.715713 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d89f2a-7d70-4cec-b878-68df2d85693a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:36.732974 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-35d89f2a-7d70-4cec-b878-68df2d85693a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:36.734155 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-35d89f2a-7d70-4cec-b878-68df2d85693a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:36.769181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:36.808005 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 160/320] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:36.830946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4caea9-2571-4a76-8769-367db4f652f8 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:36.853911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:36.936596 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4caea9-2571-4a76-8769-367db4f652f8 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:36.969918 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22eab2e-47fc-45f6-854c-d49d3936a48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:36.970675 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22eab2e-47fc-45f6-854c-d49d3936a48a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:36.970964 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22eab2e-47fc-45f6-854c-d49d3936a48a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:36.971243 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22eab2e-47fc-45f6-854c-d49d3936a48a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:36.981931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22eab2e-47fc-45f6-854c-d49d3936a48a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:36.989506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-a22eab2e-47fc-45f6-854c-d49d3936a48a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:36.989506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:36.989506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:36.989506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:36.989506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:36.989506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:36.989506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:36.989506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:36.989506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:36.989506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:36.989506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:36.989506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:36.989506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:36.989506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:36.989506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:36.989506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:36.990354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:36.990354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:36.990354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:36.990354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:36.990354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:36.990354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:36.990354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:36.990354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:36.990354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:36.990354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:36.990354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:36.990354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:36.990354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:36.990354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:36.990354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:36.990354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:36.991493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:36.991493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:36.991493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:36.991493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:36.991493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:36.991493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:36.991493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:36.991493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:36.991493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:36.991493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:36.991493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:36.991493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:36.994905 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 161/321] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:36.996402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d89f2a-7d70-4cec-b878-68df2d85693a None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:37.096268 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d89f2a-7d70-4cec-b878-68df2d85693a None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:37.134764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a03b23-b4a9-4830-b597-4ba5e53502ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:37.135445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a03b23-b4a9-4830-b597-4ba5e53502ef None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:37.135728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a03b23-b4a9-4830-b597-4ba5e53502ef None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:37.136004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a03b23-b4a9-4830-b597-4ba5e53502ef None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:37.136750 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a03b23-b4a9-4830-b597-4ba5e53502ef None admin] RBAC: Adding query filter params (name=member) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:37.137059 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a03b23-b4a9-4830-b597-4ba5e53502ef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:37.138436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a03b23-b4a9-4830-b597-4ba5e53502ef None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:37.147657 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 161/322] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:37.153213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22eab2e-47fc-45f6-854c-d49d3936a48a None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:37.177314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc9a42-8990-4f6f-af3c-37e21fbeee29 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:37.177618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc9a42-8990-4f6f-af3c-37e21fbeee29 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:37.177896 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfc9a42-8990-4f6f-af3c-37e21fbeee29 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:37.179886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 162/323] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:47:37.185010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6540011-0061-404b-8957-4184e986fc3d None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:37.185420 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6540011-0061-404b-8957-4184e986fc3d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:37.185704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6540011-0061-404b-8957-4184e986fc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:37.202856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a6540011-0061-404b-8957-4184e986fc3d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:37.204046 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a6540011-0061-404b-8957-4184e986fc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:37.237593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:37.241789 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:37.268337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22eab2e-47fc-45f6-854c-d49d3936a48a None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:37.286680 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 163/324] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:47:37.306935 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f732d5c-2977-4cbc-bf32-af519dd4afa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:37.307933 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f732d5c-2977-4cbc-bf32-af519dd4afa6 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:37.308223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f732d5c-2977-4cbc-bf32-af519dd4afa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:37.308504 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f732d5c-2977-4cbc-bf32-af519dd4afa6 None admin] PATH_INFO: `/v3/users/6bd7eecb16ed429f8a651a245706a341` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:37.308659 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6540011-0061-404b-8957-4184e986fc3d None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:37.309756 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f732d5c-2977-4cbc-bf32-af519dd4afa6 None admin] RBAC: Authorizing `identity:get_user(user_id=6bd7eecb16ed429f8a651a245706a341)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:37.311301 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f732d5c-2977-4cbc-bf32-af519dd4afa6 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:37.317549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 162/325] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:37 2022] GET /identity/v3/users/6bd7eecb16ed429f8a651a245706a341 => generated 293 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:37.318540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f732d5c-2977-4cbc-bf32-af519dd4afa6 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:37.408505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6540011-0061-404b-8957-4184e986fc3d None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:37.419615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f732d5c-2977-4cbc-bf32-af519dd4afa6 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:37.442159 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fcf14b-cb36-4159-babb-399945e5803d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:37.442986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcf14b-cb36-4159-babb-399945e5803d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:37.443255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcf14b-cb36-4159-babb-399945e5803d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:37.443530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fcf14b-cb36-4159-babb-399945e5803d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:37.444692 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcf14b-cb36-4159-babb-399945e5803d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:37.446111 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fcf14b-cb36-4159-babb-399945e5803d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:37.447757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 164/326] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:47:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:37.448895 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43975ef2-40f8-4241-9dc2-56b69b3f80a9 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:37.449179 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43975ef2-40f8-4241-9dc2-56b69b3f80a9 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:37.449455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43975ef2-40f8-4241-9dc2-56b69b3f80a9 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:37.450684 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 165/327] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:37 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:37.455328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43975ef2-40f8-4241-9dc2-56b69b3f80a9 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:37.455445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd92b06-97ac-4137-8d4d-b48e4e079db6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:37.455990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd92b06-97ac-4137-8d4d-b48e4e079db6 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:37.456278 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd92b06-97ac-4137-8d4d-b48e4e079db6 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:37.456555 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd92b06-97ac-4137-8d4d-b48e4e079db6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:37.457388 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd92b06-97ac-4137-8d4d-b48e4e079db6 None admin] RBAC: Adding query filter params (role.id=bff4f694fb57451e9a0783dc014187f2, user.id=6bd7eecb16ed429f8a651a245706a341, scope.system=all) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:37.457834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd92b06-97ac-4137-8d4d-b48e4e079db6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:37.459170 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd92b06-97ac-4137-8d4d-b48e4e079db6 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:37.471876 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 163/328] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jun 16 22:47:37 2022] GET /identity/v3/role_assignments?user.id=6bd7eecb16ed429f8a651a245706a341&scope.system=all&role.id=bff4f694fb57451e9a0783dc014187f2 => generated 510 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:37.473201 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5c1cb-1a51-43ff-b141-a23a3af6f975 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:37.473495 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5c1cb-1a51-43ff-b141-a23a3af6f975 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:37.473771 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb5c1cb-1a51-43ff-b141-a23a3af6f975 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:37.487106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:37.493951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:37.497503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-adb5c1cb-1a51-43ff-b141-a23a3af6f975 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:37.499009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-adb5c1cb-1a51-43ff-b141-a23a3af6f975 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:37.578267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 164/329] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:37.587673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43975ef2-40f8-4241-9dc2-56b69b3f80a9 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:37.604558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5c1cb-1a51-43ff-b141-a23a3af6f975 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:37.625139 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177689ff-314b-4ffc-bde0-221559ee633a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:37.625827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177689ff-314b-4ffc-bde0-221559ee633a None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:37.626116 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177689ff-314b-4ffc-bde0-221559ee633a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:37.626397 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177689ff-314b-4ffc-bde0-221559ee633a None admin] PATH_INFO: `/v3/groups` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:37.627135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177689ff-314b-4ffc-bde0-221559ee633a None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:37.628573 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177689ff-314b-4ffc-bde0-221559ee633a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:37.647407 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 166/330] 10.222.0.17 () {60 vars in 1134 bytes} [Thu Jun 16 22:47:37 2022] POST /identity/v3/groups => generated 226 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:47:37.724010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb5c1cb-1a51-43ff-b141-a23a3af6f975 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:37.760540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c3d81-204f-4b27-937a-8de695933331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:37.761262 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c3d81-204f-4b27-937a-8de695933331 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:37.761542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c3d81-204f-4b27-937a-8de695933331 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:37.761817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c3d81-204f-4b27-937a-8de695933331 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:37.764719 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:37.773799 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c3d81-204f-4b27-937a-8de695933331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:37.781741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-d31c3d81-204f-4b27-937a-8de695933331 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 22:47:37.781741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:37.781741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:37.781741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:37.781741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:37.781741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:37.781741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:37.781741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:37.781741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:37.781741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:37.781741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:37.781741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:37.781741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:37.781741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:37.781741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:37.781741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:37.782798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:37.782798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:37.782798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:37.782798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:37.782798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:37.782798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:37.782798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:37.782798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:37.782798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:37.782798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:37.782798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:37.782798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:37.782798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:37.782798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:37.782798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:37.782798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:37.783580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:37.783580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:37.783580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:37.783580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:37.783580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:37.783580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:37.783580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:37.783580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:37.783580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:37.783580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:37.783580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 22:47:37.783580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:37.783580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 165/331] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:47:37.789796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177689ff-314b-4ffc-bde0-221559ee633a None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:37.882288 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:37.893918 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177689ff-314b-4ffc-bde0-221559ee633a None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:37.939999 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c087d9-1a8e-43c5-823c-ea9643b6023d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:37.940989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c087d9-1a8e-43c5-823c-ea9643b6023d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:37.941459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c087d9-1a8e-43c5-823c-ea9643b6023d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:37.941733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c087d9-1a8e-43c5-823c-ea9643b6023d None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:37.942484 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c087d9-1a8e-43c5-823c-ea9643b6023d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:37.942833 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c087d9-1a8e-43c5-823c-ea9643b6023d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:37.944285 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c087d9-1a8e-43c5-823c-ea9643b6023d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:37.954023 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 167/332] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:37.960190 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c3d81-204f-4b27-937a-8de695933331 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:38.069114 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c3d81-204f-4b27-937a-8de695933331 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:38.103941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da30bed-3d34-4533-8719-df3793a0164d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:38.104828 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da30bed-3d34-4533-8719-df3793a0164d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:38.105115 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da30bed-3d34-4533-8719-df3793a0164d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:38.105395 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da30bed-3d34-4533-8719-df3793a0164d None admin] PATH_INFO: `/v3/users/90b4403d4c9e4966acbd173bd8e8255a` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:38.106621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da30bed-3d34-4533-8719-df3793a0164d None admin] RBAC: Authorizing `identity:get_user(user_id=90b4403d4c9e4966acbd173bd8e8255a)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:38.108081 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da30bed-3d34-4533-8719-df3793a0164d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:38.109716 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 166/333] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:37 2022] GET /identity/v3/users/90b4403d4c9e4966acbd173bd8e8255a => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:38.115399 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c087d9-1a8e-43c5-823c-ea9643b6023d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:38.235528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c087d9-1a8e-43c5-823c-ea9643b6023d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:38.274833 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264732a0-06f1-4c65-8a9b-4cef6b9896e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:38.275715 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264732a0-06f1-4c65-8a9b-4cef6b9896e5 None admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:38.276000 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264732a0-06f1-4c65-8a9b-4cef6b9896e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:38.276278 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264732a0-06f1-4c65-8a9b-4cef6b9896e5 None admin] PATH_INFO: `/v3/system/users/90b4403d4c9e4966acbd173bd8e8255a/roles/8eefae9314f544de97883fa2b10a2046` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:38.278154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264732a0-06f1-4c65-8a9b-4cef6b9896e5 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=90b4403d4c9e4966acbd173bd8e8255a, role_id=8eefae9314f544de97883fa2b10a2046)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:38.279590 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264732a0-06f1-4c65-8a9b-4cef6b9896e5 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:38.296800 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 168/334] 10.222.0.17 () {58 vars in 1334 bytes} [Thu Jun 16 22:47:38 2022] PUT /identity/v3/system/users/90b4403d4c9e4966acbd173bd8e8255a/roles/8eefae9314f544de97883fa2b10a2046 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:47:40.776165 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b2828-7ab3-4d48-a773-7e49bab6565b None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:40.777174 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b2828-7ab3-4d48-a773-7e49bab6565b None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:40.777697 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b2828-7ab3-4d48-a773-7e49bab6565b None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:40.779785 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 167/335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:40.785551 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6281c05-46da-43e0-b494-ed7e6b127435 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:40.786395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6281c05-46da-43e0-b494-ed7e6b127435 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:40.786910 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6281c05-46da-43e0-b494-ed7e6b127435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:40.794065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c70124-396a-42e3-9f7a-42091848ada1 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:40.795242 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c70124-396a-42e3-9f7a-42091848ada1 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:40.795754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c70124-396a-42e3-9f7a-42091848ada1 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:40.797328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 168/336] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:47:40.802765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908b182-59cf-4e75-84ea-22856c37a6cf None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:40.803339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908b182-59cf-4e75-84ea-22856c37a6cf None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:40.803837 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a908b182-59cf-4e75-84ea-22856c37a6cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:40.828693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-a908b182-59cf-4e75-84ea-22856c37a6cf None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:40.829917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-f6281c05-46da-43e0-b494-ed7e6b127435 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:40.831357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-a908b182-59cf-4e75-84ea-22856c37a6cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:40.832700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-f6281c05-46da-43e0-b494-ed7e6b127435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:40.866970 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:40.951105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 169/337] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:47:40.958537 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:40.967899 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 169/338] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:47:40.987704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6281c05-46da-43e0-b494-ed7e6b127435 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:41.010936 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908b182-59cf-4e75-84ea-22856c37a6cf None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:41.167532 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6281c05-46da-43e0-b494-ed7e6b127435 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:41.193114 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a908b182-59cf-4e75-84ea-22856c37a6cf None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:41.215038 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e698d-3fcc-449d-b3b3-01ca252041e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:41.216170 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e698d-3fcc-449d-b3b3-01ca252041e1 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:41.216628 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e698d-3fcc-449d-b3b3-01ca252041e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:41.217041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e698d-3fcc-449d-b3b3-01ca252041e1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:41.236443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e698d-3fcc-449d-b3b3-01ca252041e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:41.248793 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ef36b9-b628-467f-a6e7-6d992dfdf85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:41.249271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-b18e698d-3fcc-449d-b3b3-01ca252041e1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:41.249271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:41.249271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:41.249271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:41.249271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:41.249271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:41.249271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:41.249271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:41.249271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:41.249271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:41.249271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:41.249271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:41.249271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:41.249271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:41.249271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:41.249271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:41.250108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:41.250108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:41.250108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:41.250108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:41.250108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:41.250108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:41.250108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:41.250108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:41.250108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:41.250108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:41.250108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:41.250108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:41.250108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:41.250108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:41.250108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:41.250108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:41.255080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:41.255080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:41.255080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:41.255080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:41.255080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:41.255080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:41.255080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:41.255080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:41.255080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:41.255080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:41.255080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:41.255080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:41.255080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 170/339] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:47:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:41.255080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b20e77-99d6-4409-856c-1f21a3e91783 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:41.255788 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b20e77-99d6-4409-856c-1f21a3e91783 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:41.255788 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b20e77-99d6-4409-856c-1f21a3e91783 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:41.255788 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 171/340] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:41.260031 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef36b9-b628-467f-a6e7-6d992dfdf85a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:41.260330 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef36b9-b628-467f-a6e7-6d992dfdf85a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:41.260608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ef36b9-b628-467f-a6e7-6d992dfdf85a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:41.262353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcda563-52cb-4fcb-a09f-816987d4fe32 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:41.262874 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcda563-52cb-4fcb-a09f-816987d4fe32 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:41.263136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcda563-52cb-4fcb-a09f-816987d4fe32 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:41.264531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 172/341] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:41 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:41.265251 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcda563-52cb-4fcb-a09f-816987d4fe32 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:41.276453 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97ef36b9-b628-467f-a6e7-6d992dfdf85a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:41.278754 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:41.288818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-97ef36b9-b628-467f-a6e7-6d992dfdf85a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 22:47:41.288818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:41.288818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:41.288818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:41.288818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:41.288818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:41.288818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:41.288818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:41.288818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:41.288818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:41.288818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:41.288818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:41.288818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:41.288818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:41.288818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:41.288818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:41.289698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:41.289698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:41.289698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:41.289698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:41.289698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:41.289698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:41.289698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:41.289698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:41.289698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:41.289698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:41.289698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:41.289698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:41.289698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:41.289698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:41.289698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:41.289698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:41.290471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:41.290471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:41.290471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:41.290471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:41.290471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:41.290471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:41.290471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:41.290471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:41.290471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:41.290471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:41.290471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 22:47:41.290471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:41.292283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 170/342] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:47:41.295646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784130c6-1e7a-458d-861c-9663196fe9b4 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:41.295646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784130c6-1e7a-458d-861c-9663196fe9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:41.295646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784130c6-1e7a-458d-861c-9663196fe9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:41.319004 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-784130c6-1e7a-458d-861c-9663196fe9b4 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:41.320511 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-784130c6-1e7a-458d-861c-9663196fe9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:41.383894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:41.397117 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcda563-52cb-4fcb-a09f-816987d4fe32 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:41.403929 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 171/343] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:41.405567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f76fcb-7365-4dbd-83c7-ae928be0a728 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:41.406000 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f76fcb-7365-4dbd-83c7-ae928be0a728 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:41.406419 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f76fcb-7365-4dbd-83c7-ae928be0a728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:41.429015 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-e8f76fcb-7365-4dbd-83c7-ae928be0a728 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:41.431121 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-e8f76fcb-7365-4dbd-83c7-ae928be0a728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:41.434922 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaca569-3a0a-49b9-90db-a5fcb95dbf3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:41.435599 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaca569-3a0a-49b9-90db-a5fcb95dbf3f None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:41.435888 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaca569-3a0a-49b9-90db-a5fcb95dbf3f None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:41.436166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaca569-3a0a-49b9-90db-a5fcb95dbf3f None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:41.436920 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaca569-3a0a-49b9-90db-a5fcb95dbf3f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:41.437231 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaca569-3a0a-49b9-90db-a5fcb95dbf3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:41.438647 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaca569-3a0a-49b9-90db-a5fcb95dbf3f None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:41.448592 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 173/344] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:47:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:41.449243 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaca569-3a0a-49b9-90db-a5fcb95dbf3f None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:41.526313 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 172/345] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:41.527713 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb138e6f-3020-4c7e-9579-6cf932428adf None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:41.528378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb138e6f-3020-4c7e-9579-6cf932428adf None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:41.528659 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb138e6f-3020-4c7e-9579-6cf932428adf None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:41.529396 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:41.530988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 173/346] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:41.531649 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb138e6f-3020-4c7e-9579-6cf932428adf None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:41.576250 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaca569-3a0a-49b9-90db-a5fcb95dbf3f None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:41.625546 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49def630-0d1b-4114-8249-fb951cbb72e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:41.626232 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49def630-0d1b-4114-8249-fb951cbb72e0 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:41.626523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49def630-0d1b-4114-8249-fb951cbb72e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:41.626837 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49def630-0d1b-4114-8249-fb951cbb72e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:41.627599 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49def630-0d1b-4114-8249-fb951cbb72e0 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:41.627911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49def630-0d1b-4114-8249-fb951cbb72e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:41.629288 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49def630-0d1b-4114-8249-fb951cbb72e0 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:41.639595 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 174/347] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:41.640425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49def630-0d1b-4114-8249-fb951cbb72e0 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:41.675506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb138e6f-3020-4c7e-9579-6cf932428adf None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:41.694893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:41.723684 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10f2ae6-cbbf-4021-a5a1-69d0322950bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:41.727114 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f2ae6-cbbf-4021-a5a1-69d0322950bd None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:41.727400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f2ae6-cbbf-4021-a5a1-69d0322950bd None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:41.727674 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10f2ae6-cbbf-4021-a5a1-69d0322950bd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:41.739926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10f2ae6-cbbf-4021-a5a1-69d0322950bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:41.748307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-a10f2ae6-cbbf-4021-a5a1-69d0322950bd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:41.748307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:41.748307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:41.748307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:41.748307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:41.748307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:41.748307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:41.748307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:41.748307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:41.748307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:41.748307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:41.748307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:41.748307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:41.748307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:41.748307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:41.748307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:41.749202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:41.749202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:41.749202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:41.749202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:41.749202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:41.749202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:41.749202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:41.749202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:41.749202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:41.749202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:41.749202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:41.749202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:41.749202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:41.749202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:41.749202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:41.749202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:41.750065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:41.750065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:41.750065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:41.750065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:41.750065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:41.750065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:41.750065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:41.750065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:41.750065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:41.750065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:41.750065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:41.750065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:41.750065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 174/348] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:41.751378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572a6ef-0468-4cda-9fea-25db510d3a1a None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:41.751378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572a6ef-0468-4cda-9fea-25db510d3a1a None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:41.751590 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f572a6ef-0468-4cda-9fea-25db510d3a1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:41.775352 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-f572a6ef-0468-4cda-9fea-25db510d3a1a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:41.776897 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f572a6ef-0468-4cda-9fea-25db510d3a1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:41.780985 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49def630-0d1b-4114-8249-fb951cbb72e0 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:41.819336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:41.827769 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f19ce-5960-4400-a31a-733defbd9478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:41.828439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f19ce-5960-4400-a31a-733defbd9478 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:41.828781 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f19ce-5960-4400-a31a-733defbd9478 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:41.829067 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f19ce-5960-4400-a31a-733defbd9478 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:41.858188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 175/349] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:41.859550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572a6ef-0468-4cda-9fea-25db510d3a1a None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:41.915547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f19ce-5960-4400-a31a-733defbd9478 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:41.916820 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f19ce-5960-4400-a31a-733defbd9478 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:41.996533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f572a6ef-0468-4cda-9fea-25db510d3a1a None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:42.000485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:42.039855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 175/350] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:42.041021 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f19ce-5960-4400-a31a-733defbd9478 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:42.048028 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88b22c-3469-42d6-be0a-77c9523890a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:42.049484 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88b22c-3469-42d6-be0a-77c9523890a9 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:42.049827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88b22c-3469-42d6-be0a-77c9523890a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:42.050167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a88b22c-3469-42d6-be0a-77c9523890a9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:42.063124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a88b22c-3469-42d6-be0a-77c9523890a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:42.073539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-4a88b22c-3469-42d6-be0a-77c9523890a9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:42.073539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:42.073539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:42.073539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:42.073539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:42.073539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:42.073539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:42.073539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:42.073539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:42.073539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:42.073539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:42.073539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:42.073539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:42.073539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:42.073539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:42.073539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:42.081297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:42.081297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:42.081297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:42.081297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:42.081297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:42.081297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:42.081297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:42.081297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:42.081297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:42.081297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:42.081297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:42.081297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:42.081297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:42.081297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:42.081297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:42.081297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:42.082219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:42.082219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:42.082219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:42.082219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:42.082219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:42.082219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:42.082219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:42.082219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:42.082219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:42.082219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:42.082219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:42.082219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:42.082219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 176/351] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:47:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:42.082219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88b22c-3469-42d6-be0a-77c9523890a9 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:42.088626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:42.203898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f19ce-5960-4400-a31a-733defbd9478 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:42.211616 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a88b22c-3469-42d6-be0a-77c9523890a9 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:42.248786 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83c287-dbe8-4440-98b6-729c6038322b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:42.249596 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83c287-dbe8-4440-98b6-729c6038322b None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:42.249882 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83c287-dbe8-4440-98b6-729c6038322b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:42.250306 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f83c287-dbe8-4440-98b6-729c6038322b None admin] PATH_INFO: `/v3/users/90b4403d4c9e4966acbd173bd8e8255a` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:42.251377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f83c287-dbe8-4440-98b6-729c6038322b None admin] RBAC: Authorizing `identity:get_user(user_id=90b4403d4c9e4966acbd173bd8e8255a)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:42.253438 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f83c287-dbe8-4440-98b6-729c6038322b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:42.254915 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae2b25-d89a-42d4-80f8-9224cfe1a80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:42.255492 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 176/352] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:47:42 2022] GET /identity/v3/users/90b4403d4c9e4966acbd173bd8e8255a => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:42.255674 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae2b25-d89a-42d4-80f8-9224cfe1a80c None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:42.255953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae2b25-d89a-42d4-80f8-9224cfe1a80c None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:42.256232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aae2b25-d89a-42d4-80f8-9224cfe1a80c None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:42.256859 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83c287-dbe8-4440-98b6-729c6038322b None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:42.257552 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae2b25-d89a-42d4-80f8-9224cfe1a80c None admin] RBAC: Adding query filter params (name=member) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:42.257926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae2b25-d89a-42d4-80f8-9224cfe1a80c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:42.259339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aae2b25-d89a-42d4-80f8-9224cfe1a80c None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:42.268453 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 177/353] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:42.270456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae2b25-d89a-42d4-80f8-9224cfe1a80c None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:42.303181 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:42.376590 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f83c287-dbe8-4440-98b6-729c6038322b None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:42.378133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:42.402265 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aae2b25-d89a-42d4-80f8-9224cfe1a80c None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:42.419766 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e884c65-1a32-4a9a-a71a-57464e1fb27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:42.420974 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e884c65-1a32-4a9a-a71a-57464e1fb27e None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:42.421262 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e884c65-1a32-4a9a-a71a-57464e1fb27e None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:42.421542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e884c65-1a32-4a9a-a71a-57464e1fb27e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:42.434921 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e884c65-1a32-4a9a-a71a-57464e1fb27e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:42.444765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-6e884c65-1a32-4a9a-a71a-57464e1fb27e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:42.444765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:42.444765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:42.444765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:42.444765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:42.444765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:42.444765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:42.444765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:42.444765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:42.444765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:42.444765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:42.444765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:42.444765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:42.444765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:42.444765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:42.444765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:42.445700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:42.445700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:42.445700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:42.445700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:42.445700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:42.445700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:42.445700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:42.445700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:42.445700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:42.445700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:42.445700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:42.445700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:42.445700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:42.445700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:42.445700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:42.445700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:42.446525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:42.446525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:42.446525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:42.446525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:42.446525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:42.446525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:42.446525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:42.446525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:42.446525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:42.446525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:42.446525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:42.446525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:42.447248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e722ba4-0392-4373-a76e-1c4fdc4b9cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:42.447248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e722ba4-0392-4373-a76e-1c4fdc4b9cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:42.447402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 177/354] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:42.447478 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e722ba4-0392-4373-a76e-1c4fdc4b9cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:42.447478 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e722ba4-0392-4373-a76e-1c4fdc4b9cc0 None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:42.447606 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e884c65-1a32-4a9a-a71a-57464e1fb27e None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:42.451510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e722ba4-0392-4373-a76e-1c4fdc4b9cc0 None admin] RBAC: Authorizing `identity:get_project(project_id=f39c31704e364effbc1f39d76b609302)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:42.453244 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e722ba4-0392-4373-a76e-1c4fdc4b9cc0 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:42.456047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 178/355] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:42 2022] GET /identity/v3/projects/f39c31704e364effbc1f39d76b609302 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:42.457308 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aebf0df-ab06-4afa-9979-b6e05d354e25 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:42.457596 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aebf0df-ab06-4afa-9979-b6e05d354e25 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:42.457875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aebf0df-ab06-4afa-9979-b6e05d354e25 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:42.463170 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 179/356] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:42.463170 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aebf0df-ab06-4afa-9979-b6e05d354e25 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:42.568567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:42.569555 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e884c65-1a32-4a9a-a71a-57464e1fb27e None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:42.577120 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aebf0df-ab06-4afa-9979-b6e05d354e25 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:42.610744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a403795-f0d8-4d90-afb7-47a92223cc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:42.611423 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a403795-f0d8-4d90-afb7-47a92223cc41 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:42.611713 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a403795-f0d8-4d90-afb7-47a92223cc41 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:42.612038 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a403795-f0d8-4d90-afb7-47a92223cc41 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:42.612803 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a403795-f0d8-4d90-afb7-47a92223cc41 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:42.613115 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a403795-f0d8-4d90-afb7-47a92223cc41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:42.614577 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a403795-f0d8-4d90-afb7-47a92223cc41 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:42.619241 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d0a10-d2c8-4c77-8c76-a729e5307bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:42.619399 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d0a10-d2c8-4c77-8c76-a729e5307bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:42.620115 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d0a10-d2c8-4c77-8c76-a729e5307bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:42.620115 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699d0a10-d2c8-4c77-8c76-a729e5307bc6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:42.620759 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d0a10-d2c8-4c77-8c76-a729e5307bc6 None admin] RBAC: Adding query filter params (role.id=8eefae9314f544de97883fa2b10a2046, user.id=90b4403d4c9e4966acbd173bd8e8255a, scope.system=all) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:42.621520 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d0a10-d2c8-4c77-8c76-a729e5307bc6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:42.623551 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-699d0a10-d2c8-4c77-8c76-a729e5307bc6 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:42.626659 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 178/357] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:47:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:42.626659 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a403795-f0d8-4d90-afb7-47a92223cc41 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:42.638649 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 180/358] 10.222.0.17 () {56 vars in 1308 bytes} [Thu Jun 16 22:47:42 2022] GET /identity/v3/role_assignments?user.id=90b4403d4c9e4966acbd173bd8e8255a&scope.system=all&role.id=8eefae9314f544de97883fa2b10a2046 => generated 510 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:42.639554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d0a10-d2c8-4c77-8c76-a729e5307bc6 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:42.649192 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:42.764760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a403795-f0d8-4d90-afb7-47a92223cc41 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:42.797350 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699d0a10-d2c8-4c77-8c76-a729e5307bc6 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:42.813286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac823185-bf4f-43c6-93d7-46825ebcf369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:42.813945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac823185-bf4f-43c6-93d7-46825ebcf369 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:42.814236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac823185-bf4f-43c6-93d7-46825ebcf369 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:42.814516 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac823185-bf4f-43c6-93d7-46825ebcf369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:42.841488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07718e0e-ffa5-47a7-94fd-921f0fbcd9f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:42.842103 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07718e0e-ffa5-47a7-94fd-921f0fbcd9f7 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:42.842386 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07718e0e-ffa5-47a7-94fd-921f0fbcd9f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:42.842771 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07718e0e-ffa5-47a7-94fd-921f0fbcd9f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:42.843519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07718e0e-ffa5-47a7-94fd-921f0fbcd9f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:42.843826 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07718e0e-ffa5-47a7-94fd-921f0fbcd9f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:42.845205 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07718e0e-ffa5-47a7-94fd-921f0fbcd9f7 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:42.854336 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 181/359] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:42.859214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e2298-5286-4ab4-a911-0cc3faa2b1e8 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:42.859594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e2298-5286-4ab4-a911-0cc3faa2b1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:42.859881 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92e2298-5286-4ab4-a911-0cc3faa2b1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:42.879848 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-a92e2298-5286-4ab4-a911-0cc3faa2b1e8 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:42.881125 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-a92e2298-5286-4ab4-a911-0cc3faa2b1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:42.893982 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:42.906373 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac823185-bf4f-43c6-93d7-46825ebcf369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:42.908523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac823185-bf4f-43c6-93d7-46825ebcf369 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:42.935768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:42.966241 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 182/360] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:42.967740 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e2298-5286-4ab4-a911-0cc3faa2b1e8 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:43.041948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 179/361] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:43.042930 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac823185-bf4f-43c6-93d7-46825ebcf369 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:43.089070 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92e2298-5286-4ab4-a911-0cc3faa2b1e8 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:43.133084 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81df10f-4176-4878-b93b-87d953493435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:43.134442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81df10f-4176-4878-b93b-87d953493435 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:43.134442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81df10f-4176-4878-b93b-87d953493435 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:43.134442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81df10f-4176-4878-b93b-87d953493435 None admin] PATH_INFO: `/v3/groups/2fc0a3a06f37449d8ab42381c30ef043` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:43.140771 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81df10f-4176-4878-b93b-87d953493435 None admin] RBAC: Authorizing `identity:get_group(group_id=2fc0a3a06f37449d8ab42381c30ef043)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:43.142360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81df10f-4176-4878-b93b-87d953493435 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:43.144143 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 183/362] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 22:47:42 2022] GET /identity/v3/groups/2fc0a3a06f37449d8ab42381c30ef043 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:43.146339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81df10f-4176-4878-b93b-87d953493435 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:43.157117 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:43.159850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac823185-bf4f-43c6-93d7-46825ebcf369 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:43.183829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:43.200850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bacb6-a535-4661-b209-e08a7373f319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:43.201868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bacb6-a535-4661-b209-e08a7373f319 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:43.201868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bacb6-a535-4661-b209-e08a7373f319 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:43.202047 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718bacb6-a535-4661-b209-e08a7373f319 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:43.263761 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81df10f-4176-4878-b93b-87d953493435 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:43.291806 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718bacb6-a535-4661-b209-e08a7373f319 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:43.293276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718bacb6-a535-4661-b209-e08a7373f319 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:43.305915 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d74acd-8b7d-47b4-8f05-fdea989c5a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:43.306573 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d74acd-8b7d-47b4-8f05-fdea989c5a3d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:43.307000 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d74acd-8b7d-47b4-8f05-fdea989c5a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:43.307280 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d74acd-8b7d-47b4-8f05-fdea989c5a3d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:43.373092 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d74acd-8b7d-47b4-8f05-fdea989c5a3d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:43.374564 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d74acd-8b7d-47b4-8f05-fdea989c5a3d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:43.384153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:43.389761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 180/363] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:43.390623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bacb6-a535-4661-b209-e08a7373f319 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:43.413775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:43.474063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 184/364] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:43.474934 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d74acd-8b7d-47b4-8f05-fdea989c5a3d None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:43.489972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718bacb6-a535-4661-b209-e08a7373f319 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:43.523977 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6b87d-430d-47ac-b38e-21ea53779682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:43.524780 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6b87d-430d-47ac-b38e-21ea53779682 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:43.525068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6b87d-430d-47ac-b38e-21ea53779682 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:43.525350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb6b87d-430d-47ac-b38e-21ea53779682 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:43.535839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb6b87d-430d-47ac-b38e-21ea53779682 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:43.543949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-abb6b87d-430d-47ac-b38e-21ea53779682 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:47:43.543949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:43.543949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:43.543949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:43.543949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:43.543949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:43.543949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:43.543949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:43.543949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:43.543949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:43.543949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:43.543949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:43.543949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:43.543949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:43.543949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:43.543949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:43.544962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:43.544962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:43.544962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:43.544962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:43.544962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:43.544962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:43.544962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:43.544962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:43.544962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:43.544962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:43.544962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:43.544962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:43.544962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:43.544962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:43.544962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:43.544962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:43.545936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:43.545936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:43.545936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:43.545936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:43.545936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:43.545936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:43.545936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:43.545936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:43.545936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:43.545936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:43.545936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:47:43.545936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:43.545936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 181/365] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:47:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:43.546830 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6b87d-430d-47ac-b38e-21ea53779682 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:43.582377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d74acd-8b7d-47b4-8f05-fdea989c5a3d None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:43.592134 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:43.617295 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946bcdf-421d-4888-99a8-5360acf054de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:43.618145 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946bcdf-421d-4888-99a8-5360acf054de None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:43.618427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946bcdf-421d-4888-99a8-5360acf054de None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:43.618740 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946bcdf-421d-4888-99a8-5360acf054de None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:43.619917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946bcdf-421d-4888-99a8-5360acf054de None admin] RBAC: Authorizing `identity:get_project(project_id=9bdf56f0db064649a642e4d566966738)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:43.621457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946bcdf-421d-4888-99a8-5360acf054de None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:43.623176 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 185/366] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:43 2022] GET /identity/v3/projects/9bdf56f0db064649a642e4d566966738 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:43.623931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946bcdf-421d-4888-99a8-5360acf054de None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:43.628269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:43.647481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb6b87d-430d-47ac-b38e-21ea53779682 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:43.680850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd72a1f-cf6b-4331-8636-45f9903b43dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:43.681506 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd72a1f-cf6b-4331-8636-45f9903b43dd None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:43.681794 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd72a1f-cf6b-4331-8636-45f9903b43dd None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:43.682081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd72a1f-cf6b-4331-8636-45f9903b43dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:43.682945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd72a1f-cf6b-4331-8636-45f9903b43dd None admin] RBAC: Adding query filter params (group.id=2fc0a3a06f37449d8ab42381c30ef043, role.id=bda1c48ca2534bc6b2ca4f3ff479b143, scope.project.id=f39c31704e364effbc1f39d76b609302) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:43.683265 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd72a1f-cf6b-4331-8636-45f9903b43dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:43.684709 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd72a1f-cf6b-4331-8636-45f9903b43dd None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:43.690053 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 182/367] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 22:47:43 2022] GET /identity/v3/role_assignments?group.id=2fc0a3a06f37449d8ab42381c30ef043&scope.project.id=f39c31704e364effbc1f39d76b609302&role.id=bda1c48ca2534bc6b2ca4f3ff479b143 => generated 260 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:43.690852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd72a1f-cf6b-4331-8636-45f9903b43dd None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:43.727984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946bcdf-421d-4888-99a8-5360acf054de None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:43.768447 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aacb7d-ed51-43ee-b39d-7e2c40b80f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:43.769325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aacb7d-ed51-43ee-b39d-7e2c40b80f0b None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:43.769606 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aacb7d-ed51-43ee-b39d-7e2c40b80f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:43.769990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aacb7d-ed51-43ee-b39d-7e2c40b80f0b None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:43.771313 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aacb7d-ed51-43ee-b39d-7e2c40b80f0b None admin] RBAC: Authorizing `identity:get_project(project_id=9bdf56f0db064649a642e4d566966738)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:43.773350 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aacb7d-ed51-43ee-b39d-7e2c40b80f0b None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:43.780085 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 186/368] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:43 2022] GET /identity/v3/projects/9bdf56f0db064649a642e4d566966738 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:43.780085 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aacb7d-ed51-43ee-b39d-7e2c40b80f0b None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:43.818486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd72a1f-cf6b-4331-8636-45f9903b43dd None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:43.830282 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:43.860079 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa904a6-0c7e-4f21-9074-4398f9c083cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:43.860956 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa904a6-0c7e-4f21-9074-4398f9c083cd None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:43.861244 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa904a6-0c7e-4f21-9074-4398f9c083cd None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:43.861523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa904a6-0c7e-4f21-9074-4398f9c083cd None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:43.862750 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa904a6-0c7e-4f21-9074-4398f9c083cd None admin] RBAC: Authorizing `identity:get_project(project_id=f39c31704e364effbc1f39d76b609302)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:43.864180 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa904a6-0c7e-4f21-9074-4398f9c083cd None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:43.870371 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 183/369] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:43 2022] GET /identity/v3/projects/f39c31704e364effbc1f39d76b609302 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:43.870371 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa904a6-0c7e-4f21-9074-4398f9c083cd None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:43.886364 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:43.892017 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aacb7d-ed51-43ee-b39d-7e2c40b80f0b None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:43.928087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d699ef6-7bf6-46b9-819e-35bab841c3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:43.929206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d699ef6-7bf6-46b9-819e-35bab841c3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:43.929494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d699ef6-7bf6-46b9-819e-35bab841c3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:43.929770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d699ef6-7bf6-46b9-819e-35bab841c3e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:43.930512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d699ef6-7bf6-46b9-819e-35bab841c3e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:43.930850 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d699ef6-7bf6-46b9-819e-35bab841c3e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:43.932223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d699ef6-7bf6-46b9-819e-35bab841c3e5 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:43.940861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 187/370] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:43.941669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d699ef6-7bf6-46b9-819e-35bab841c3e5 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:43.989847 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa904a6-0c7e-4f21-9074-4398f9c083cd None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:44.038573 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a4f97-7e29-4168-8617-a693e7c0f2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:44.039756 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a4f97-7e29-4168-8617-a693e7c0f2dd None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:44.040092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a4f97-7e29-4168-8617-a693e7c0f2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:44.040379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52a4f97-7e29-4168-8617-a693e7c0f2dd None admin] PATH_INFO: `/v3/groups/2fc0a3a06f37449d8ab42381c30ef043` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:44.042035 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a4f97-7e29-4168-8617-a693e7c0f2dd None admin] RBAC: Authorizing `identity:get_group(group_id=2fc0a3a06f37449d8ab42381c30ef043)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:44.043511 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52a4f97-7e29-4168-8617-a693e7c0f2dd None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:44.045595 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 184/371] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 22:47:43 2022] GET /identity/v3/groups/2fc0a3a06f37449d8ab42381c30ef043 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:44.046359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a4f97-7e29-4168-8617-a693e7c0f2dd None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:44.063524 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d699ef6-7bf6-46b9-819e-35bab841c3e5 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:44.073955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:44.129169 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f938f-0c5f-4347-805a-58e9c437a5d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:44.130165 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f938f-0c5f-4347-805a-58e9c437a5d8 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:44.130539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f938f-0c5f-4347-805a-58e9c437a5d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:44.130916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627f938f-0c5f-4347-805a-58e9c437a5d8 None admin] PATH_INFO: `/v3/groups/2fc0a3a06f37449d8ab42381c30ef043` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:44.132788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f938f-0c5f-4347-805a-58e9c437a5d8 None admin] RBAC: Authorizing `identity:get_group(group_id=2fc0a3a06f37449d8ab42381c30ef043)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:44.134301 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627f938f-0c5f-4347-805a-58e9c437a5d8 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:44.141204 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 188/372] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 22:47:43 2022] GET /identity/v3/groups/2fc0a3a06f37449d8ab42381c30ef043 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:44.142185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f938f-0c5f-4347-805a-58e9c437a5d8 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:44.162830 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:44.187771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52a4f97-7e29-4168-8617-a693e7c0f2dd None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:44.231443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e843b5-5cbe-4f21-b5e2-dd4f472ab503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:44.231443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e843b5-5cbe-4f21-b5e2-dd4f472ab503 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:44.231443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e843b5-5cbe-4f21-b5e2-dd4f472ab503 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:44.231443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e843b5-5cbe-4f21-b5e2-dd4f472ab503 None admin] PATH_INFO: `/v3/groups/2fc0a3a06f37449d8ab42381c30ef043` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:44.234031 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e843b5-5cbe-4f21-b5e2-dd4f472ab503 None admin] RBAC: Authorizing `identity:get_group(group_id=2fc0a3a06f37449d8ab42381c30ef043)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:44.235527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e843b5-5cbe-4f21-b5e2-dd4f472ab503 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:44.242836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 185/373] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 22:47:44 2022] GET /identity/v3/groups/2fc0a3a06f37449d8ab42381c30ef043 => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:44.242836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e843b5-5cbe-4f21-b5e2-dd4f472ab503 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:44.282472 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627f938f-0c5f-4347-805a-58e9c437a5d8 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:44.320036 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b242cf7-ba20-464f-9c56-09a34c3698c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:44.320969 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b242cf7-ba20-464f-9c56-09a34c3698c5 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:44.321330 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b242cf7-ba20-464f-9c56-09a34c3698c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:44.321683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b242cf7-ba20-464f-9c56-09a34c3698c5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:44.364074 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e843b5-5cbe-4f21-b5e2-dd4f472ab503 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:44.386174 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:44.415285 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3531d-4b77-4fe7-890c-70f664eab985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:44.416089 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b242cf7-ba20-464f-9c56-09a34c3698c5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:44.416631 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3531d-4b77-4fe7-890c-70f664eab985 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:44.417249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3531d-4b77-4fe7-890c-70f664eab985 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:44.417557 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b242cf7-ba20-464f-9c56-09a34c3698c5 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:44.418030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c3531d-4b77-4fe7-890c-70f664eab985 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:44.419318 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3531d-4b77-4fe7-890c-70f664eab985 None admin] RBAC: Adding query filter params (group.id=2fc0a3a06f37449d8ab42381c30ef043, role.id=bff4f694fb57451e9a0783dc014187f2, scope.project.id=9bdf56f0db064649a642e4d566966738) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:44.419869 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3531d-4b77-4fe7-890c-70f664eab985 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:44.421718 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c3531d-4b77-4fe7-890c-70f664eab985 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:44.428191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 186/374] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 22:47:44 2022] GET /identity/v3/role_assignments?group.id=2fc0a3a06f37449d8ab42381c30ef043&scope.project.id=9bdf56f0db064649a642e4d566966738&role.id=bff4f694fb57451e9a0783dc014187f2 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:44.439397 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3531d-4b77-4fe7-890c-70f664eab985 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:44.460267 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:44.556687 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 189/375] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:44.557935 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b242cf7-ba20-464f-9c56-09a34c3698c5 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:44.589609 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c3531d-4b77-4fe7-890c-70f664eab985 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:44.635341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f0593-48cf-4910-adef-f15387166722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:44.636011 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f0593-48cf-4910-adef-f15387166722 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:44.636302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f0593-48cf-4910-adef-f15387166722 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:44.636589 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f0593-48cf-4910-adef-f15387166722 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:44.637425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f0593-48cf-4910-adef-f15387166722 None admin] RBAC: Adding query filter params (group.id=2fc0a3a06f37449d8ab42381c30ef043, role.id=bda1c48ca2534bc6b2ca4f3ff479b143, scope.project.id=9bdf56f0db064649a642e4d566966738) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:44.637745 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f0593-48cf-4910-adef-f15387166722 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:44.639262 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f0593-48cf-4910-adef-f15387166722 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:44.650746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 187/376] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 22:47:44 2022] GET /identity/v3/role_assignments?group.id=2fc0a3a06f37449d8ab42381c30ef043&scope.project.id=9bdf56f0db064649a642e4d566966738&role.id=bda1c48ca2534bc6b2ca4f3ff479b143 => generated 260 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:44.651823 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f0593-48cf-4910-adef-f15387166722 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:44.657168 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:44.687390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b242cf7-ba20-464f-9c56-09a34c3698c5 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:44.728794 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b7586-e769-45ab-89c9-a65d04d92c08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:44.729391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b7586-e769-45ab-89c9-a65d04d92c08 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:44.729677 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b7586-e769-45ab-89c9-a65d04d92c08 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:44.729955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b7586-e769-45ab-89c9-a65d04d92c08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:44.731069 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b7586-e769-45ab-89c9-a65d04d92c08 None admin] RBAC: Adding query filter params (group.id=2fc0a3a06f37449d8ab42381c30ef043, role.id=bff4f694fb57451e9a0783dc014187f2, scope.project.id=f39c31704e364effbc1f39d76b609302) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:44.731435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b7586-e769-45ab-89c9-a65d04d92c08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:44.733104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b7586-e769-45ab-89c9-a65d04d92c08 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:44.740265 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 190/377] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 22:47:44 2022] GET /identity/v3/role_assignments?group.id=2fc0a3a06f37449d8ab42381c30ef043&scope.project.id=f39c31704e364effbc1f39d76b609302&role.id=bff4f694fb57451e9a0783dc014187f2 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:44.803034 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:44.829558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f0593-48cf-4910-adef-f15387166722 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:44.877930 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b316a-28e9-4ab9-97de-dd3777602fc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:44.878877 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b316a-28e9-4ab9-97de-dd3777602fc7 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:44.879164 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b316a-28e9-4ab9-97de-dd3777602fc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:44.879442 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7b316a-28e9-4ab9-97de-dd3777602fc7 None admin] PATH_INFO: `/v3/projects/2ffbb7bcd828426c857e7fce279519f7` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:44.880722 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b316a-28e9-4ab9-97de-dd3777602fc7 None admin] RBAC: Authorizing `identity:get_project(project_id=2ffbb7bcd828426c857e7fce279519f7)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:44.882336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7b316a-28e9-4ab9-97de-dd3777602fc7 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:44.884140 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 188/378] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:44 2022] GET /identity/v3/projects/2ffbb7bcd828426c857e7fce279519f7 => generated 344 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:44.889963 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b7586-e769-45ab-89c9-a65d04d92c08 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:45.000457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b7586-e769-45ab-89c9-a65d04d92c08 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:45.035622 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b02d3-59d5-4de1-a56d-bd9a51d7b185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:45.036531 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b02d3-59d5-4de1-a56d-bd9a51d7b185 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:45.036820 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b02d3-59d5-4de1-a56d-bd9a51d7b185 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:45.037096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b02d3-59d5-4de1-a56d-bd9a51d7b185 None admin] PATH_INFO: `/v3/groups/0dc79dd1f4674c7face18abdc242b9d4` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:45.042529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b02d3-59d5-4de1-a56d-bd9a51d7b185 None admin] RBAC: Authorizing `identity:get_group(group_id=0dc79dd1f4674c7face18abdc242b9d4)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:45.043990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b02d3-59d5-4de1-a56d-bd9a51d7b185 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:45.045926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 191/379] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 22:47:44 2022] GET /identity/v3/groups/0dc79dd1f4674c7face18abdc242b9d4 => generated 229 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:45.051781 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b316a-28e9-4ab9-97de-dd3777602fc7 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:45.176734 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7b316a-28e9-4ab9-97de-dd3777602fc7 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:45.211675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc95ecd9-2340-4a6a-8343-395d9fa88358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:45.218614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95ecd9-2340-4a6a-8343-395d9fa88358 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:45.218614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95ecd9-2340-4a6a-8343-395d9fa88358 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:45.218614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95ecd9-2340-4a6a-8343-395d9fa88358 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:45.218614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95ecd9-2340-4a6a-8343-395d9fa88358 None admin] RBAC: Adding query filter params (group.id=0dc79dd1f4674c7face18abdc242b9d4, role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.project.id=2ffbb7bcd828426c857e7fce279519f7) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:45.218614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95ecd9-2340-4a6a-8343-395d9fa88358 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:45.218614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc95ecd9-2340-4a6a-8343-395d9fa88358 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:45.221527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 189/380] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 22:47:45 2022] GET /identity/v3/role_assignments?group.id=0dc79dd1f4674c7face18abdc242b9d4&scope.project.id=2ffbb7bcd828426c857e7fce279519f7&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 260 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:46.123012 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f53748-635d-4b18-b071-a3ce8fc62d5e None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:46.123547 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f53748-635d-4b18-b071-a3ce8fc62d5e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:46.123547 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f53748-635d-4b18-b071-a3ce8fc62d5e None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:46.125319 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 192/381] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:46.130852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a80c1a-287b-4223-9ff7-15df659b3bdc None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:46.131904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a80c1a-287b-4223-9ff7-15df659b3bdc None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:46.132015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a80c1a-287b-4223-9ff7-15df659b3bdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:46.155621 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-03a80c1a-287b-4223-9ff7-15df659b3bdc None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:46.156809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-03a80c1a-287b-4223-9ff7-15df659b3bdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:46.159037 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:46.231188 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 190/382] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:46.263360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f53748-635d-4b18-b071-a3ce8fc62d5e None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:46.266274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:46.377318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f53748-635d-4b18-b071-a3ce8fc62d5e None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:46.412058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f93df-e4b4-47ad-926a-5fa2099cb532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:46.413005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f93df-e4b4-47ad-926a-5fa2099cb532 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:46.413289 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f93df-e4b4-47ad-926a-5fa2099cb532 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:46.413565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5f93df-e4b4-47ad-926a-5fa2099cb532 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:46.424310 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e5f93df-e4b4-47ad-926a-5fa2099cb532 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:46.432925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-5e5f93df-e4b4-47ad-926a-5fa2099cb532 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:46.432925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:46.432925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:46.432925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:46.432925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:46.432925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:46.432925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:46.432925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:46.432925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:46.432925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:46.432925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:46.432925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:46.432925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:46.432925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:46.432925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:46.432925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:46.433784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:46.433784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:46.433784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:46.433784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:46.433784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:46.433784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:46.433784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:46.433784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:46.433784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:46.433784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:46.433784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:46.433784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:46.433784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:46.433784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:46.433784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:46.433784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:46.434582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:46.434582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:46.434582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:46.434582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:46.434582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:46.434582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:46.434582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:46.434582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:46.434582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:46.434582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:46.434582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:46.434582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:46.434582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 193/383] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:47:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:46.439931 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a80c1a-287b-4223-9ff7-15df659b3bdc None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:46.578263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a80c1a-287b-4223-9ff7-15df659b3bdc None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:46.625706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7830d-ebae-4b63-8b69-4a1742826ebb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:46.626849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7830d-ebae-4b63-8b69-4a1742826ebb None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:46.627388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7830d-ebae-4b63-8b69-4a1742826ebb None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:46.627870 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d7830d-ebae-4b63-8b69-4a1742826ebb None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:46.628815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7830d-ebae-4b63-8b69-4a1742826ebb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:46.629326 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7830d-ebae-4b63-8b69-4a1742826ebb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:46.630966 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d7830d-ebae-4b63-8b69-4a1742826ebb None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:46.638737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:46.642021 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 191/384] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:47:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:46.647976 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f93df-e4b4-47ad-926a-5fa2099cb532 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:46.744677 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:46.770167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5f93df-e4b4-47ad-926a-5fa2099cb532 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:46.804699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28ebce-ad6f-455b-a3bb-e867435b047d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:46.805562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28ebce-ad6f-455b-a3bb-e867435b047d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:46.805843 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28ebce-ad6f-455b-a3bb-e867435b047d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:46.806114 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c28ebce-ad6f-455b-a3bb-e867435b047d None admin] PATH_INFO: `/v3/groups/2fc0a3a06f37449d8ab42381c30ef043` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:46.807305 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28ebce-ad6f-455b-a3bb-e867435b047d None admin] RBAC: Authorizing `identity:get_group(group_id=2fc0a3a06f37449d8ab42381c30ef043)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:46.808732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c28ebce-ad6f-455b-a3bb-e867435b047d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:46.810658 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 194/385] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 22:47:46 2022] GET /identity/v3/groups/2fc0a3a06f37449d8ab42381c30ef043 => generated 226 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:46.815697 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7830d-ebae-4b63-8b69-4a1742826ebb None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:46.955375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d7830d-ebae-4b63-8b69-4a1742826ebb None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:47.012551 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0221a8a5-d8aa-4bf3-aae3-364a5bea373e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:47.013442 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221a8a5-d8aa-4bf3-aae3-364a5bea373e None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:47.013729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221a8a5-d8aa-4bf3-aae3-364a5bea373e None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:47.014001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0221a8a5-d8aa-4bf3-aae3-364a5bea373e None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:47.015235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221a8a5-d8aa-4bf3-aae3-364a5bea373e None admin] RBAC: Authorizing `identity:get_project(project_id=f39c31704e364effbc1f39d76b609302)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:47.016666 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0221a8a5-d8aa-4bf3-aae3-364a5bea373e None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:47.018441 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 192/386] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:46 2022] GET /identity/v3/projects/f39c31704e364effbc1f39d76b609302 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:47.024575 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28ebce-ad6f-455b-a3bb-e867435b047d None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:47.051625 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba8825-a7fe-4392-8c5d-8d3e6e149cc9 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:47.052021 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba8825-a7fe-4392-8c5d-8d3e6e149cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:47.052321 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbba8825-a7fe-4392-8c5d-8d3e6e149cc9 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:47.053742 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 193/387] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:47.059044 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1109ec-ec80-4669-8b66-18d34689da6c None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:47.059594 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1109ec-ec80-4669-8b66-18d34689da6c None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:47.060074 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1109ec-ec80-4669-8b66-18d34689da6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:47.089572 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-7d1109ec-ec80-4669-8b66-18d34689da6c None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:47.089572 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7d1109ec-ec80-4669-8b66-18d34689da6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:47.213673 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c28ebce-ad6f-455b-a3bb-e867435b047d None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:47.215713 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:47.229029 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 194/388] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:47:47.252266 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:47.262876 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1109ec-ec80-4669-8b66-18d34689da6c None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:47.272350 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df8fd43-d88b-4aab-968d-3f6d9ecb3700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:47.273263 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8fd43-d88b-4aab-968d-3f6d9ecb3700 None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:47.273549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8fd43-d88b-4aab-968d-3f6d9ecb3700 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:47.273829 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df8fd43-d88b-4aab-968d-3f6d9ecb3700 None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302/groups/2fc0a3a06f37449d8ab42381c30ef043/roles/bda1c48ca2534bc6b2ca4f3ff479b143` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:47.280468 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df8fd43-d88b-4aab-968d-3f6d9ecb3700 None admin] RBAC: Authorizing `identity:create_grant(project_id=f39c31704e364effbc1f39d76b609302, group_id=2fc0a3a06f37449d8ab42381c30ef043, role_id=bda1c48ca2534bc6b2ca4f3ff479b143)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:47.281923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df8fd43-d88b-4aab-968d-3f6d9ecb3700 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:47.313955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 195/389] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jun 16 22:47:47 2022] PUT /identity/v3/projects/f39c31704e364effbc1f39d76b609302/groups/2fc0a3a06f37449d8ab42381c30ef043/roles/bda1c48ca2534bc6b2ca4f3ff479b143 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:47:47.391606 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1109ec-ec80-4669-8b66-18d34689da6c None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:47.428907 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ac5a5-27f5-4003-9d36-52cfade0f4dd None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:47.429480 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ac5a5-27f5-4003-9d36-52cfade0f4dd None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:47.429906 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9ac5a5-27f5-4003-9d36-52cfade0f4dd None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:47.431893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 196/390] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:47.438648 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa522e7-68e1-4dad-a65c-2a91fc4074f6 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:47.438944 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa522e7-68e1-4dad-a65c-2a91fc4074f6 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:47.439229 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa522e7-68e1-4dad-a65c-2a91fc4074f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:47.444051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a597baa-0d61-4cd2-b6fc-ff4b29b36378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:47.444921 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a597baa-0d61-4cd2-b6fc-ff4b29b36378 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:47.445215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a597baa-0d61-4cd2-b6fc-ff4b29b36378 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:47.445549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a597baa-0d61-4cd2-b6fc-ff4b29b36378 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:47.471843 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a597baa-0d61-4cd2-b6fc-ff4b29b36378 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:47.475916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-eaa522e7-68e1-4dad-a65c-2a91fc4074f6 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:47.477141 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-eaa522e7-68e1-4dad-a65c-2a91fc4074f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:47.481771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-2a597baa-0d61-4cd2-b6fc-ff4b29b36378 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:47.481771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:47.481771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:47.481771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:47.481771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:47.481771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:47.481771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:47.481771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:47.481771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:47.481771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:47.481771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:47.481771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:47.481771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:47.481771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:47.481771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:47.481771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:47.482773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:47.482773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:47.482773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:47.482773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:47.482773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:47.482773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:47.482773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:47.482773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:47.482773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:47.482773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:47.482773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:47.482773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:47.482773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:47.482773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:47.482773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:47.482773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:47.483548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:47.483548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:47.483548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:47.483548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:47.483548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:47.483548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:47.483548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:47.483548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:47.483548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:47.483548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:47.483548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:47.483548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:47.483548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 195/391] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:47:47.484912 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606b0fc-bf8c-4c0d-a0b6-04aeff0101c3 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:47.486867 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606b0fc-bf8c-4c0d-a0b6-04aeff0101c3 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:47.486867 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4606b0fc-bf8c-4c0d-a0b6-04aeff0101c3 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:47.487256 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 196/392] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:47.488879 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606b0fc-bf8c-4c0d-a0b6-04aeff0101c3 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:47.563639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:47.586020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 197/393] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:47.587773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca9800-5b0f-41fb-9fb9-4dd679b5d843 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:47.588067 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca9800-5b0f-41fb-9fb9-4dd679b5d843 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:47.588351 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fca9800-5b0f-41fb-9fb9-4dd679b5d843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:47.619072 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-2fca9800-5b0f-41fb-9fb9-4dd679b5d843 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:47.620547 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-2fca9800-5b0f-41fb-9fb9-4dd679b5d843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:47.632448 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4606b0fc-bf8c-4c0d-a0b6-04aeff0101c3 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:47.670856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e2771-48f0-456c-9ea3-a64d004202e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:47.671602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e2771-48f0-456c-9ea3-a64d004202e6 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:47.671900 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e2771-48f0-456c-9ea3-a64d004202e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:47.672179 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5e2771-48f0-456c-9ea3-a64d004202e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:47.672979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e2771-48f0-456c-9ea3-a64d004202e6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:47.673293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e2771-48f0-456c-9ea3-a64d004202e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:47.674776 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5e2771-48f0-456c-9ea3-a64d004202e6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:47.689136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 197/394] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:47.690126 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e2771-48f0-456c-9ea3-a64d004202e6 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:47.711077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 198/395] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:47.712035 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca9800-5b0f-41fb-9fb9-4dd679b5d843 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:47.774276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:47.824323 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5e2771-48f0-456c-9ea3-a64d004202e6 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:47.846198 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:47.850008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fca9800-5b0f-41fb-9fb9-4dd679b5d843 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:47.863242 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52016190-f409-49c9-9f53-72ccc5d4c199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:47.864109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52016190-f409-49c9-9f53-72ccc5d4c199 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:47.864395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52016190-f409-49c9-9f53-72ccc5d4c199 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:47.864630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52016190-f409-49c9-9f53-72ccc5d4c199 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:47.875829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52016190-f409-49c9-9f53-72ccc5d4c199 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:47.883832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-52016190-f409-49c9-9f53-72ccc5d4c199 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:47.883832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:47.883832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:47.883832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:47.883832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:47.883832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:47.883832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:47.883832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:47.883832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:47.883832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:47.883832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:47.883832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:47.883832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:47.883832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:47.883832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:47.883832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:47.885853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:47.885853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:47.885853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:47.885853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:47.885853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:47.885853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:47.885853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:47.885853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:47.885853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:47.885853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:47.885853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:47.885853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:47.885853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:47.885853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:47.885853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:47.885853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:47.887809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:47.887809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:47.887809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:47.887809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:47.887809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:47.887809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:47.887809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:47.887809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:47.887809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:47.887809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:47.887809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:47.887809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:47.890716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 198/396] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:47.891067 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52016190-f409-49c9-9f53-72ccc5d4c199 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:47.892563 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c18fa-d02b-4aae-a0d4-e560a70cb7fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:47.893629 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c18fa-d02b-4aae-a0d4-e560a70cb7fe None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:47.894079 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c18fa-d02b-4aae-a0d4-e560a70cb7fe None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:47.894537 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64c18fa-d02b-4aae-a0d4-e560a70cb7fe None admin] PATH_INFO: `/v3/groups/2fc0a3a06f37449d8ab42381c30ef043` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:47.897634 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c18fa-d02b-4aae-a0d4-e560a70cb7fe None admin] RBAC: Authorizing `identity:get_group(group_id=2fc0a3a06f37449d8ab42381c30ef043)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:47.899356 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64c18fa-d02b-4aae-a0d4-e560a70cb7fe None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:47.901733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 199/397] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 22:47:47 2022] GET /identity/v3/groups/2fc0a3a06f37449d8ab42381c30ef043 => generated 226 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:47.902877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c18fa-d02b-4aae-a0d4-e560a70cb7fe None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:48.020365 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52016190-f409-49c9-9f53-72ccc5d4c199 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:48.032902 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64c18fa-d02b-4aae-a0d4-e560a70cb7fe None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:48.055912 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:48.057117 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e87eb-f5ae-49ad-9220-3e6a521d530d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:48.058205 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e87eb-f5ae-49ad-9220-3e6a521d530d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:48.058491 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e87eb-f5ae-49ad-9220-3e6a521d530d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:48.058793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e87eb-f5ae-49ad-9220-3e6a521d530d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:48.069555 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e87eb-f5ae-49ad-9220-3e6a521d530d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:48.076232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6396cb-8787-4535-8a82-42a639d00b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:48.077596 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6396cb-8787-4535-8a82-42a639d00b14 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:48.077772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-599e87eb-f5ae-49ad-9220-3e6a521d530d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:48.077772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:48.077772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:48.077772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:48.077772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:48.077772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:48.077772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:48.077772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:48.077772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:48.077772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:48.077772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:48.077772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:48.077772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:48.077772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:48.077772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:48.077772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:48.079316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:48.079316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:48.079316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:48.079316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:48.079316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:48.079316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:48.079316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:48.079316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:48.079316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:48.079316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:48.079316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:48.079316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:48.079316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:48.079316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:48.079316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:48.079316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:48.081092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:48.081092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:48.081092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:48.081092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:48.081092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:48.081092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:48.081092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:48.081092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:48.081092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:48.081092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:48.081092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:48.081092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:48.081092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 199/398] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:47:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:48.081769 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6396cb-8787-4535-8a82-42a639d00b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:48.081769 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6396cb-8787-4535-8a82-42a639d00b14 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:48.081769 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6396cb-8787-4535-8a82-42a639d00b14 None admin] RBAC: Adding query filter params (name=member) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:48.081769 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6396cb-8787-4535-8a82-42a639d00b14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:48.082381 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6396cb-8787-4535-8a82-42a639d00b14 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:48.083607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e87eb-f5ae-49ad-9220-3e6a521d530d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:48.092518 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 200/399] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:48.093999 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fbba4-9fb5-43ff-854b-dec3976287d8 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:48.094111 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fbba4-9fb5-43ff-854b-dec3976287d8 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:48.094397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351fbba4-9fb5-43ff-854b-dec3976287d8 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:48.099550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 201/400] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:48.100856 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fbba4-9fb5-43ff-854b-dec3976287d8 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:48.134278 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:48.211290 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e87eb-f5ae-49ad-9220-3e6a521d530d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:48.223902 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351fbba4-9fb5-43ff-854b-dec3976287d8 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:48.253238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd419cc-1c03-45e5-89f9-ba39f4dbb815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:48.254119 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd419cc-1c03-45e5-89f9-ba39f4dbb815 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:48.254409 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd419cc-1c03-45e5-89f9-ba39f4dbb815 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:48.254723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd419cc-1c03-45e5-89f9-ba39f4dbb815 None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:48.255938 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd419cc-1c03-45e5-89f9-ba39f4dbb815 None admin] RBAC: Authorizing `identity:get_project(project_id=9bdf56f0db064649a642e4d566966738)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:48.257435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd419cc-1c03-45e5-89f9-ba39f4dbb815 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:48.259367 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 200/401] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:48 2022] GET /identity/v3/projects/9bdf56f0db064649a642e4d566966738 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:48.259929 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd419cc-1c03-45e5-89f9-ba39f4dbb815 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:48.269017 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccc5bee-8001-4425-a377-1d4774f3618d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:48.270118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc5bee-8001-4425-a377-1d4774f3618d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:48.270415 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc5bee-8001-4425-a377-1d4774f3618d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:48.270733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccc5bee-8001-4425-a377-1d4774f3618d None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:48.271586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc5bee-8001-4425-a377-1d4774f3618d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:48.271900 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc5bee-8001-4425-a377-1d4774f3618d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:48.273725 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccc5bee-8001-4425-a377-1d4774f3618d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:48.285265 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 202/402] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:47:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:48.286518 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2b107-bcf8-42aa-aa5b-9e574144863f None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:48.286846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2b107-bcf8-42aa-aa5b-9e574144863f None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:48.287135 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b2b107-bcf8-42aa-aa5b-9e574144863f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:48.305504 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-c3b2b107-bcf8-42aa-aa5b-9e574144863f None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:48.306738 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c3b2b107-bcf8-42aa-aa5b-9e574144863f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:48.346457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:48.378154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd419cc-1c03-45e5-89f9-ba39f4dbb815 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:48.394732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 203/403] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:48.395195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:48.396150 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2b107-bcf8-42aa-aa5b-9e574144863f None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:48.436972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff5a4c-5e72-440f-ae14-9bead6626a8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:48.437960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff5a4c-5e72-440f-ae14-9bead6626a8b None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:48.438247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff5a4c-5e72-440f-ae14-9bead6626a8b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:48.438525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ff5a4c-5e72-440f-ae14-9bead6626a8b None admin] PATH_INFO: `/v3/groups/2fc0a3a06f37449d8ab42381c30ef043` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:48.439859 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff5a4c-5e72-440f-ae14-9bead6626a8b None admin] RBAC: Authorizing `identity:get_group(group_id=2fc0a3a06f37449d8ab42381c30ef043)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:48.441386 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ff5a4c-5e72-440f-ae14-9bead6626a8b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:48.443198 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 201/404] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 22:47:48 2022] GET /identity/v3/groups/2fc0a3a06f37449d8ab42381c30ef043 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:48.448665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff5a4c-5e72-440f-ae14-9bead6626a8b None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:48.507904 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b2b107-bcf8-42aa-aa5b-9e574144863f None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:48.545839 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921388a5-6605-4e6c-b59b-31c065e572d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:48.546677 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921388a5-6605-4e6c-b59b-31c065e572d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:48.546963 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921388a5-6605-4e6c-b59b-31c065e572d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:48.547236 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921388a5-6605-4e6c-b59b-31c065e572d4 None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738/groups/2fc0a3a06f37449d8ab42381c30ef043/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:48.549212 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921388a5-6605-4e6c-b59b-31c065e572d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bdf56f0db064649a642e4d566966738, group_id=2fc0a3a06f37449d8ab42381c30ef043, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:48.550686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921388a5-6605-4e6c-b59b-31c065e572d4 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:48.563639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ff5a4c-5e72-440f-ae14-9bead6626a8b None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:48.574907 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 204/405] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jun 16 22:47:48 2022] PUT /identity/v3/projects/9bdf56f0db064649a642e4d566966738/groups/2fc0a3a06f37449d8ab42381c30ef043/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:47:48.574907 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921388a5-6605-4e6c-b59b-31c065e572d4 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:48.603423 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa803d8-bd2d-415a-82f7-ebaac45ae127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:48.604357 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa803d8-bd2d-415a-82f7-ebaac45ae127 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:48.604681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa803d8-bd2d-415a-82f7-ebaac45ae127 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:48.604963 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa803d8-bd2d-415a-82f7-ebaac45ae127 None admin] PATH_INFO: `/v3/groups/2fc0a3a06f37449d8ab42381c30ef043` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:48.606313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa803d8-bd2d-415a-82f7-ebaac45ae127 None admin] RBAC: Authorizing `identity:get_group(group_id=2fc0a3a06f37449d8ab42381c30ef043)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:48.607822 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa803d8-bd2d-415a-82f7-ebaac45ae127 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:48.609619 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 202/406] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 22:47:48 2022] GET /identity/v3/groups/2fc0a3a06f37449d8ab42381c30ef043 => generated 226 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:48.610393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa803d8-bd2d-415a-82f7-ebaac45ae127 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:48.630629 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:48.668759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:48.731874 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921388a5-6605-4e6c-b59b-31c065e572d4 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:48.734004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa803d8-bd2d-415a-82f7-ebaac45ae127 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:48.780445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83092f-3f02-47fa-bcba-7fb8ab321816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:48.781260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83092f-3f02-47fa-bcba-7fb8ab321816 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:48.781547 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83092f-3f02-47fa-bcba-7fb8ab321816 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:48.781823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83092f-3f02-47fa-bcba-7fb8ab321816 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:48.786305 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca8bb7-7459-4cb1-b68d-beeafbe4800f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:48.786505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca8bb7-7459-4cb1-b68d-beeafbe4800f None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:48.786839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca8bb7-7459-4cb1-b68d-beeafbe4800f None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:48.787124 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eca8bb7-7459-4cb1-b68d-beeafbe4800f None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:48.788966 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca8bb7-7459-4cb1-b68d-beeafbe4800f None admin] RBAC: Authorizing `identity:get_project(project_id=f39c31704e364effbc1f39d76b609302)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:48.790660 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eca8bb7-7459-4cb1-b68d-beeafbe4800f None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:48.792326 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 203/407] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:48 2022] GET /identity/v3/projects/f39c31704e364effbc1f39d76b609302 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:48.797198 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca8bb7-7459-4cb1-b68d-beeafbe4800f None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:48.802873 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a83092f-3f02-47fa-bcba-7fb8ab321816 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:48.811387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-9a83092f-3f02-47fa-bcba-7fb8ab321816 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:47:48.811387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:48.811387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:48.811387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:48.811387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:48.811387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:48.811387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:48.811387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:48.811387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:48.811387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:48.811387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:48.811387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:48.811387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:48.811387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:48.811387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:48.811387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:48.812553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:48.812553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:48.812553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:48.812553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:48.812553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:48.812553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:48.812553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:48.812553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:48.812553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:48.812553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:48.812553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:48.812553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:48.812553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:48.812553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:48.812553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:48.812553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:48.814104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:48.814104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:48.814104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:48.814104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:48.814104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:48.814104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:48.814104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:48.814104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:48.814104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:48.814104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:48.814104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:47:48.814104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:48.815627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 205/408] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:47:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:48.816414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83092f-3f02-47fa-bcba-7fb8ab321816 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:48.920883 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:48.923630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eca8bb7-7459-4cb1-b68d-beeafbe4800f None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:48.929854 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83092f-3f02-47fa-bcba-7fb8ab321816 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:48.946619 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:48.970163 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226e943-cc22-4ca4-a6e9-4afbbea2c804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:48.971069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226e943-cc22-4ca4-a6e9-4afbbea2c804 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:48.971353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226e943-cc22-4ca4-a6e9-4afbbea2c804 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:48.971633 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2226e943-cc22-4ca4-a6e9-4afbbea2c804 None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:48.972860 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226e943-cc22-4ca4-a6e9-4afbbea2c804 None admin] RBAC: Authorizing `identity:get_project(project_id=9bdf56f0db064649a642e4d566966738)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:48.974305 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2226e943-cc22-4ca4-a6e9-4afbbea2c804 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:48.976784 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 204/409] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:48 2022] GET /identity/v3/projects/9bdf56f0db064649a642e4d566966738 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:48.977312 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226e943-cc22-4ca4-a6e9-4afbbea2c804 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:48.978282 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07baa05-4618-4ef0-9f3f-39720b1c7086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:48.979208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07baa05-4618-4ef0-9f3f-39720b1c7086 None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:48.979490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07baa05-4618-4ef0-9f3f-39720b1c7086 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:48.979764 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07baa05-4618-4ef0-9f3f-39720b1c7086 None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302/groups/2fc0a3a06f37449d8ab42381c30ef043/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:48.982131 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07baa05-4618-4ef0-9f3f-39720b1c7086 None admin] RBAC: Authorizing `identity:create_grant(project_id=f39c31704e364effbc1f39d76b609302, group_id=2fc0a3a06f37449d8ab42381c30ef043, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:48.983867 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07baa05-4618-4ef0-9f3f-39720b1c7086 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:49.003146 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 206/410] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jun 16 22:47:48 2022] PUT /identity/v3/projects/f39c31704e364effbc1f39d76b609302/groups/2fc0a3a06f37449d8ab42381c30ef043/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:47:49.003962 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07baa05-4618-4ef0-9f3f-39720b1c7086 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:49.119874 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2226e943-cc22-4ca4-a6e9-4afbbea2c804 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:49.128063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07baa05-4618-4ef0-9f3f-39720b1c7086 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:49.157725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95051ce4-1113-4f58-9642-db8ab380f985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:49.158406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95051ce4-1113-4f58-9642-db8ab380f985 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:49.158738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95051ce4-1113-4f58-9642-db8ab380f985 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:49.159022 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95051ce4-1113-4f58-9642-db8ab380f985 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:49.159776 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95051ce4-1113-4f58-9642-db8ab380f985 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:49.160086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95051ce4-1113-4f58-9642-db8ab380f985 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:49.161460 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95051ce4-1113-4f58-9642-db8ab380f985 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:49.171691 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 205/411] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:49.177323 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95051ce4-1113-4f58-9642-db8ab380f985 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:49.183463 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e213bf-236d-44d1-a1a2-8a590ce6dfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:49.184398 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e213bf-236d-44d1-a1a2-8a590ce6dfb7 None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:49.184679 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e213bf-236d-44d1-a1a2-8a590ce6dfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:49.184956 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e213bf-236d-44d1-a1a2-8a590ce6dfb7 None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738/groups/2fc0a3a06f37449d8ab42381c30ef043/roles/bda1c48ca2534bc6b2ca4f3ff479b143` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:49.192248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e213bf-236d-44d1-a1a2-8a590ce6dfb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=9bdf56f0db064649a642e4d566966738, group_id=2fc0a3a06f37449d8ab42381c30ef043, role_id=bda1c48ca2534bc6b2ca4f3ff479b143)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:49.193722 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e213bf-236d-44d1-a1a2-8a590ce6dfb7 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:49.222009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 207/412] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jun 16 22:47:49 2022] PUT /identity/v3/projects/9bdf56f0db064649a642e4d566966738/groups/2fc0a3a06f37449d8ab42381c30ef043/roles/bda1c48ca2534bc6b2ca4f3ff479b143 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:47:49.230610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:49.300153 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95051ce4-1113-4f58-9642-db8ab380f985 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:49.338976 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967dd8f-fa89-4f9d-b874-22742d0e676d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:49.339860 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967dd8f-fa89-4f9d-b874-22742d0e676d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:49.340164 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967dd8f-fa89-4f9d-b874-22742d0e676d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:49.340482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5967dd8f-fa89-4f9d-b874-22742d0e676d None admin] PATH_INFO: `/v3/groups/0dc79dd1f4674c7face18abdc242b9d4` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:49.341731 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967dd8f-fa89-4f9d-b874-22742d0e676d None admin] RBAC: Authorizing `identity:get_group(group_id=0dc79dd1f4674c7face18abdc242b9d4)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:49.343222 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5967dd8f-fa89-4f9d-b874-22742d0e676d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:49.345204 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 206/413] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 22:47:49 2022] GET /identity/v3/groups/0dc79dd1f4674c7face18abdc242b9d4 => generated 229 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:49.350898 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e213bf-236d-44d1-a1a2-8a590ce6dfb7 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:49.369702 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:49.476567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e213bf-236d-44d1-a1a2-8a590ce6dfb7 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:49.511399 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837df6e5-3c21-42a5-9161-6e49c307346c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:49.512277 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837df6e5-3c21-42a5-9161-6e49c307346c None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:49.512564 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837df6e5-3c21-42a5-9161-6e49c307346c None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:49.512848 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837df6e5-3c21-42a5-9161-6e49c307346c None admin] PATH_INFO: `/v3/projects/2ffbb7bcd828426c857e7fce279519f7` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:49.514039 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837df6e5-3c21-42a5-9161-6e49c307346c None admin] RBAC: Authorizing `identity:get_project(project_id=2ffbb7bcd828426c857e7fce279519f7)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:49.515506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837df6e5-3c21-42a5-9161-6e49c307346c None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:49.517535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 208/414] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:49 2022] GET /identity/v3/projects/2ffbb7bcd828426c857e7fce279519f7 => generated 344 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:49.523474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967dd8f-fa89-4f9d-b874-22742d0e676d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:49.664916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5967dd8f-fa89-4f9d-b874-22742d0e676d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:49.691510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:49.705075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd914f86-34a4-483d-a50f-21a42a3224fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:49.705980 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd914f86-34a4-483d-a50f-21a42a3224fc None admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:49.706267 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd914f86-34a4-483d-a50f-21a42a3224fc None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:49.706549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd914f86-34a4-483d-a50f-21a42a3224fc None admin] PATH_INFO: `/v3/projects/2ffbb7bcd828426c857e7fce279519f7/groups/0dc79dd1f4674c7face18abdc242b9d4/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:49.708868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd914f86-34a4-483d-a50f-21a42a3224fc None admin] RBAC: Authorizing `identity:create_grant(project_id=2ffbb7bcd828426c857e7fce279519f7, group_id=0dc79dd1f4674c7face18abdc242b9d4, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:49.710547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd914f86-34a4-483d-a50f-21a42a3224fc None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:49.714934 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30a9f8-8e38-4291-8df4-6755a092e13b None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:49.715232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30a9f8-8e38-4291-8df4-6755a092e13b None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:49.715520 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f30a9f8-8e38-4291-8df4-6755a092e13b None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:49.717129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 209/415] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:49.722320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b552bdb-9c42-4efe-80ed-d4f02b49939c None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:49.722634 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b552bdb-9c42-4efe-80ed-d4f02b49939c None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:49.722912 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b552bdb-9c42-4efe-80ed-d4f02b49939c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:49.735961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 207/416] 10.222.0.17 () {58 vars in 1442 bytes} [Thu Jun 16 22:47:49 2022] PUT /identity/v3/projects/2ffbb7bcd828426c857e7fce279519f7/groups/0dc79dd1f4674c7face18abdc242b9d4/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:47:49.749027 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-7b552bdb-9c42-4efe-80ed-d4f02b49939c None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:49.750272 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-7b552bdb-9c42-4efe-80ed-d4f02b49939c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:49.847022 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 210/417] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:47:49.871122 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd914f86-34a4-483d-a50f-21a42a3224fc None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:49.995259 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd914f86-34a4-483d-a50f-21a42a3224fc None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:50.031352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff74e6e-531d-4a5e-9fc4-0b64618f8742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:50.032169 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff74e6e-531d-4a5e-9fc4-0b64618f8742 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:50.032458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff74e6e-531d-4a5e-9fc4-0b64618f8742 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:50.032739 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff74e6e-531d-4a5e-9fc4-0b64618f8742 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:50.043657 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff74e6e-531d-4a5e-9fc4-0b64618f8742 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:50.051698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-7ff74e6e-531d-4a5e-9fc4-0b64618f8742 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:50.051698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:50.051698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:50.051698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:50.051698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:50.051698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:50.051698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:50.051698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:50.051698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:50.051698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:50.051698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:50.051698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:50.051698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:50.051698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:50.051698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:50.051698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:50.052607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:50.052607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:50.052607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:50.052607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:50.052607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:50.052607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:50.052607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:50.052607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:50.052607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:50.052607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:50.052607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:50.052607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:50.052607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:50.052607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:50.052607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:50.052607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:50.053934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:50.053934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:50.053934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:50.053934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:50.053934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:50.053934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:50.053934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:50.053934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:50.053934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:50.053934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:50.053934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:50.053934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:50.053934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 208/418] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:47:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:50.059596 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b552bdb-9c42-4efe-80ed-d4f02b49939c None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:50.072507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:50.168614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b552bdb-9c42-4efe-80ed-d4f02b49939c None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:50.204218 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28910f6e-b269-4bb1-a3ac-e8bdf45cf0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:50.204899 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28910f6e-b269-4bb1-a3ac-e8bdf45cf0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:50.205184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28910f6e-b269-4bb1-a3ac-e8bdf45cf0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:50.205468 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28910f6e-b269-4bb1-a3ac-e8bdf45cf0a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:50.206214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28910f6e-b269-4bb1-a3ac-e8bdf45cf0a6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:50.206522 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28910f6e-b269-4bb1-a3ac-e8bdf45cf0a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:50.207954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28910f6e-b269-4bb1-a3ac-e8bdf45cf0a6 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:50.217510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 211/419] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:47:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:50.222953 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff74e6e-531d-4a5e-9fc4-0b64618f8742 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:50.255675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:50.327925 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff74e6e-531d-4a5e-9fc4-0b64618f8742 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:50.363236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf05636-122a-4a72-a167-16d0afdb2554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:50.363895 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf05636-122a-4a72-a167-16d0afdb2554 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:50.364181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf05636-122a-4a72-a167-16d0afdb2554 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:50.364468 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf05636-122a-4a72-a167-16d0afdb2554 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:50.432574 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf05636-122a-4a72-a167-16d0afdb2554 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:50.434060 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf05636-122a-4a72-a167-16d0afdb2554 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:50.467009 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:50.538477 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 209/420] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:50.543749 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28910f6e-b269-4bb1-a3ac-e8bdf45cf0a6 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:50.655187 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:50.697432 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28910f6e-b269-4bb1-a3ac-e8bdf45cf0a6 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:50.738928 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea678d-620d-4d9e-85b3-e3e0af53e491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:50.739810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea678d-620d-4d9e-85b3-e3e0af53e491 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:50.740097 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea678d-620d-4d9e-85b3-e3e0af53e491 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:50.740375 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea678d-620d-4d9e-85b3-e3e0af53e491 None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:50.742098 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea678d-620d-4d9e-85b3-e3e0af53e491 None admin] RBAC: Authorizing `identity:get_project(project_id=f39c31704e364effbc1f39d76b609302)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:50.743559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea678d-620d-4d9e-85b3-e3e0af53e491 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:50.745560 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 212/421] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:50 2022] GET /identity/v3/projects/f39c31704e364effbc1f39d76b609302 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:50.752572 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf05636-122a-4a72-a167-16d0afdb2554 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:50.873936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf05636-122a-4a72-a167-16d0afdb2554 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:50.913203 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:50.926372 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d8f38d-60e1-4503-b15b-830abcec3da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:50.927489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8f38d-60e1-4503-b15b-830abcec3da0 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:50.927922 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8f38d-60e1-4503-b15b-830abcec3da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:50.928905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d8f38d-60e1-4503-b15b-830abcec3da0 None admin] PATH_INFO: `/v3/groups/2fc0a3a06f37449d8ab42381c30ef043` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:50.934497 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d8f38d-60e1-4503-b15b-830abcec3da0 None admin] RBAC: Authorizing `identity:get_group(group_id=2fc0a3a06f37449d8ab42381c30ef043)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:50.936128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d8f38d-60e1-4503-b15b-830abcec3da0 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:50.942171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 210/422] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 22:47:50 2022] GET /identity/v3/groups/2fc0a3a06f37449d8ab42381c30ef043 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:50.943815 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea678d-620d-4d9e-85b3-e3e0af53e491 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:51.107149 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea678d-620d-4d9e-85b3-e3e0af53e491 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:51.159379 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9908885e-839f-4e46-ad12-c922ccd3a1d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:51.159973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908885e-839f-4e46-ad12-c922ccd3a1d1 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:51.160264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908885e-839f-4e46-ad12-c922ccd3a1d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:51.160547 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9908885e-839f-4e46-ad12-c922ccd3a1d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:51.161372 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908885e-839f-4e46-ad12-c922ccd3a1d1 None admin] RBAC: Adding query filter params (group.id=2fc0a3a06f37449d8ab42381c30ef043, role.id=bda1c48ca2534bc6b2ca4f3ff479b143, scope.project.id=f39c31704e364effbc1f39d76b609302) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:51.161687 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908885e-839f-4e46-ad12-c922ccd3a1d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:51.163670 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9908885e-839f-4e46-ad12-c922ccd3a1d1 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:51.169985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 213/423] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 22:47:50 2022] GET /identity/v3/role_assignments?group.id=2fc0a3a06f37449d8ab42381c30ef043&scope.project.id=f39c31704e364effbc1f39d76b609302&role.id=bda1c48ca2534bc6b2ca4f3ff479b143 => generated 611 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:51.326017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb01edd-6cca-471a-8c92-548dcf823e29 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:51.326017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb01edd-6cca-471a-8c92-548dcf823e29 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:51.326017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb01edd-6cca-471a-8c92-548dcf823e29 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:51.327281 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 211/424] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:51.332802 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c7b62-acd9-4ee5-b661-22eb86af2930 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:51.333101 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c7b62-acd9-4ee5-b661-22eb86af2930 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:51.333385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c7b62-acd9-4ee5-b661-22eb86af2930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:51.345450 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:51.365083 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-412c7b62-acd9-4ee5-b661-22eb86af2930 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:51.366411 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-412c7b62-acd9-4ee5-b661-22eb86af2930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:51.444415 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 214/425] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:51.468254 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb01edd-6cca-471a-8c92-548dcf823e29 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:51.581742 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb01edd-6cca-471a-8c92-548dcf823e29 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:51.626305 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcb26ee-2707-4924-812b-e5abff0ff291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:51.627169 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb26ee-2707-4924-812b-e5abff0ff291 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:51.627460 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb26ee-2707-4924-812b-e5abff0ff291 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:51.627740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcb26ee-2707-4924-812b-e5abff0ff291 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:51.639049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcb26ee-2707-4924-812b-e5abff0ff291 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:51.646824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-4fcb26ee-2707-4924-812b-e5abff0ff291 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:51.646824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:51.646824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:51.646824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:51.646824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:51.646824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:51.646824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:51.646824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:51.646824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:51.646824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:51.646824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:51.646824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:51.646824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:51.646824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:51.646824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:51.646824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:51.647761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:51.647761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:51.647761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:51.647761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:51.647761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:51.647761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:51.647761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:51.647761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:51.647761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:51.647761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:51.647761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:51.647761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:51.647761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:51.647761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:51.647761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:51.647761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:51.648725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:51.648725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:51.648725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:51.648725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:51.648725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:51.648725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:51.648725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:51.648725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:51.648725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:51.648725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:51.648725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:51.648725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:51.648725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 212/426] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:51.656223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c7b62-acd9-4ee5-b661-22eb86af2930 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:51.776582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c7b62-acd9-4ee5-b661-22eb86af2930 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:51.800119 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea81271-77bd-431a-8435-4cbf586622bc None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:51.800446 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea81271-77bd-431a-8435-4cbf586622bc None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:51.800682 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea81271-77bd-431a-8435-4cbf586622bc None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:51.802355 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 213/427] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:47:51.807606 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e8b81-0353-410b-9b27-36e7da9842bd None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:51.807894 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e8b81-0353-410b-9b27-36e7da9842bd None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:51.808169 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e8b81-0353-410b-9b27-36e7da9842bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:51.815009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff52872e-ef96-4c86-83f1-5f569104f85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:51.815691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52872e-ef96-4c86-83f1-5f569104f85d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:51.815976 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52872e-ef96-4c86-83f1-5f569104f85d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:51.816252 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff52872e-ef96-4c86-83f1-5f569104f85d None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:51.816994 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff52872e-ef96-4c86-83f1-5f569104f85d None admin] RBAC: Adding query filter params (name=member) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:51.817300 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff52872e-ef96-4c86-83f1-5f569104f85d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:51.818678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff52872e-ef96-4c86-83f1-5f569104f85d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:51.820078 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:51.833914 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 215/428] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:51.835398 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff52872e-ef96-4c86-83f1-5f569104f85d None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:51.835741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-945e8b81-0353-410b-9b27-36e7da9842bd None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:51.837012 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-945e8b81-0353-410b-9b27-36e7da9842bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:51.895843 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:51.923661 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 214/429] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:47:51.941353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e8b81-0353-410b-9b27-36e7da9842bd None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:51.958935 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff52872e-ef96-4c86-83f1-5f569104f85d None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:52.000214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31be33d1-3721-41ae-b740-cc126c8bffb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:52.000872 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be33d1-3721-41ae-b740-cc126c8bffb3 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:52.001152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be33d1-3721-41ae-b740-cc126c8bffb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:52.001439 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31be33d1-3721-41ae-b740-cc126c8bffb3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:52.055628 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e8b81-0353-410b-9b27-36e7da9842bd None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:52.070363 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:52.077733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be33d1-3721-41ae-b740-cc126c8bffb3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:52.079261 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31be33d1-3721-41ae-b740-cc126c8bffb3 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:52.093991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b425af-d03f-43f7-913c-1e780ebcdd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:52.094817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b425af-d03f-43f7-913c-1e780ebcdd73 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:52.095107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b425af-d03f-43f7-913c-1e780ebcdd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:52.095384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b425af-d03f-43f7-913c-1e780ebcdd73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:52.106412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b425af-d03f-43f7-913c-1e780ebcdd73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:52.114626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-02b425af-d03f-43f7-913c-1e780ebcdd73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:52.114626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:52.114626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:52.114626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:52.114626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:52.114626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:52.114626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:52.114626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:52.114626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:52.114626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:52.114626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:52.114626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:52.114626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:52.114626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:52.114626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:52.114626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:52.115496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:52.115496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:52.115496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:52.115496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:52.115496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:52.115496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:52.115496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:52.115496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:52.115496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:52.115496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:52.115496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:52.115496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:52.115496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:52.115496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:52.115496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:52.115496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:52.116935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:52.116935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:52.116935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:52.116935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:52.116935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:52.116935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:52.116935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:52.116935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:52.116935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:52.116935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:52.116935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 22:47:52.116935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:52.121004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 215/430] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:47:52.122333 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73480836-22a6-47cf-9272-0d192371b85d None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:52.122672 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73480836-22a6-47cf-9272-0d192371b85d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:52.123034 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73480836-22a6-47cf-9272-0d192371b85d None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:52.124577 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 216/431] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:52.125264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73480836-22a6-47cf-9272-0d192371b85d None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:52.187636 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 216/432] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 22:47:52.188317 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:52.193649 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0b819-001c-49e9-a559-b43ba4bb12f7 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:52.194006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0b819-001c-49e9-a559-b43ba4bb12f7 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:52.194292 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c0b819-001c-49e9-a559-b43ba4bb12f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:52.214484 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-41c0b819-001c-49e9-a559-b43ba4bb12f7 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:52.215692 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-41c0b819-001c-49e9-a559-b43ba4bb12f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:52.248837 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73480836-22a6-47cf-9272-0d192371b85d None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:52.284692 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448f22e-a26b-4a73-8fb5-efaecbaaf7bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:52.285365 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448f22e-a26b-4a73-8fb5-efaecbaaf7bb None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:52.285651 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448f22e-a26b-4a73-8fb5-efaecbaaf7bb None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:52.285937 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d448f22e-a26b-4a73-8fb5-efaecbaaf7bb None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:52.286707 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448f22e-a26b-4a73-8fb5-efaecbaaf7bb None admin] RBAC: Adding query filter params (name=member) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:52.287017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448f22e-a26b-4a73-8fb5-efaecbaaf7bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:52.289015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d448f22e-a26b-4a73-8fb5-efaecbaaf7bb None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:52.294887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 217/433] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:52.295952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0b819-001c-49e9-a559-b43ba4bb12f7 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:52.302073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 217/434] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:47:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:52.303974 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448f22e-a26b-4a73-8fb5-efaecbaaf7bb None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:52.334146 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:52.411221 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c0b819-001c-49e9-a559-b43ba4bb12f7 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:52.417536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d448f22e-a26b-4a73-8fb5-efaecbaaf7bb None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:52.428988 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:52.449614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff023d0f-4ddf-4806-9089-c87ec50f629a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:52.451258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff023d0f-4ddf-4806-9089-c87ec50f629a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:52.451691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff023d0f-4ddf-4806-9089-c87ec50f629a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:52.452132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff023d0f-4ddf-4806-9089-c87ec50f629a None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:52.453568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff023d0f-4ddf-4806-9089-c87ec50f629a None admin] RBAC: Authorizing `identity:get_project(project_id=9bdf56f0db064649a642e4d566966738)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:52.455463 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff023d0f-4ddf-4806-9089-c87ec50f629a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:52.455841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dc1c7c-2c95-4e80-bea6-1d955ed01189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:52.456518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc1c7c-2c95-4e80-bea6-1d955ed01189 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:52.456807 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc1c7c-2c95-4e80-bea6-1d955ed01189 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:52.457088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dc1c7c-2c95-4e80-bea6-1d955ed01189 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:52.457275 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 218/435] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:52 2022] GET /identity/v3/projects/9bdf56f0db064649a642e4d566966738 => generated 298 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:52.463008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff023d0f-4ddf-4806-9089-c87ec50f629a None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:52.533883 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc1c7c-2c95-4e80-bea6-1d955ed01189 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:52.535358 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dc1c7c-2c95-4e80-bea6-1d955ed01189 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:52.560692 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:52.567413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff023d0f-4ddf-4806-9089-c87ec50f629a None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:52.601798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162aebec-1b39-4d05-ac99-f745122322dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:52.602529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162aebec-1b39-4d05-ac99-f745122322dc None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:52.602844 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162aebec-1b39-4d05-ac99-f745122322dc None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:52.603125 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162aebec-1b39-4d05-ac99-f745122322dc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:52.613085 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162aebec-1b39-4d05-ac99-f745122322dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:52.621112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-162aebec-1b39-4d05-ac99-f745122322dc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:52.621112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:52.621112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:52.621112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:52.621112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:52.621112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:52.621112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:52.621112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:52.621112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:52.621112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:52.621112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:52.621112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:52.621112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:52.621112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:52.621112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:52.621112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:52.621947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:52.621947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:52.621947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:52.621947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:52.621947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:52.621947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:52.621947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:52.621947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:52.621947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:52.621947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:52.621947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:52.621947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:52.621947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:52.621947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:52.621947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:52.621947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:52.622770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:52.622770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:52.622770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:52.622770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:52.622770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:52.622770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:52.622770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:52.622770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:52.622770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:52.622770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:52.622770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:52.622770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:52.622770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 219/436] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:47:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:52.623433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162aebec-1b39-4d05-ac99-f745122322dc None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:52.639719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 218/437] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:52.639719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71227e13-8f42-4c82-8bfb-adde40f3822f None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:52.639719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71227e13-8f42-4c82-8bfb-adde40f3822f None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:52.639719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71227e13-8f42-4c82-8bfb-adde40f3822f None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:52.640785 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 219/438] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:52.641667 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71227e13-8f42-4c82-8bfb-adde40f3822f None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:52.656792 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:52.727911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162aebec-1b39-4d05-ac99-f745122322dc None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:52.746676 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71227e13-8f42-4c82-8bfb-adde40f3822f None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:52.760861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da3b32-0ad1-461c-9c0a-1512db57af42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:52.761718 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da3b32-0ad1-461c-9c0a-1512db57af42 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:52.761998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da3b32-0ad1-461c-9c0a-1512db57af42 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:52.762274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da3b32-0ad1-461c-9c0a-1512db57af42 None admin] PATH_INFO: `/v3/groups/2fc0a3a06f37449d8ab42381c30ef043` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:52.763472 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da3b32-0ad1-461c-9c0a-1512db57af42 None admin] RBAC: Authorizing `identity:get_group(group_id=2fc0a3a06f37449d8ab42381c30ef043)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:52.764897 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01da3b32-0ad1-461c-9c0a-1512db57af42 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:52.766498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 220/439] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 22:47:52 2022] GET /identity/v3/groups/2fc0a3a06f37449d8ab42381c30ef043 => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:52.771743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da3b32-0ad1-461c-9c0a-1512db57af42 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:52.781152 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f281167c-ce47-4ebd-a701-aca75f274c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:52.781825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281167c-ce47-4ebd-a701-aca75f274c39 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:52.782117 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281167c-ce47-4ebd-a701-aca75f274c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:52.782400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f281167c-ce47-4ebd-a701-aca75f274c39 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:52.783183 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281167c-ce47-4ebd-a701-aca75f274c39 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:52.783494 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281167c-ce47-4ebd-a701-aca75f274c39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:52.784850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f281167c-ce47-4ebd-a701-aca75f274c39 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:52.791751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:52.792884 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 220/440] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:47:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:52.793786 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2643d-aa45-42fc-b63e-e0f75df4c049 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:52.794084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2643d-aa45-42fc-b63e-e0f75df4c049 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:52.794522 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2643d-aa45-42fc-b63e-e0f75df4c049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:52.810052 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-e2b2643d-aa45-42fc-b63e-e0f75df4c049 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:52.811264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e2b2643d-aa45-42fc-b63e-e0f75df4c049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:52.867986 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da3b32-0ad1-461c-9c0a-1512db57af42 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:52.876556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 221/441] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:52.877361 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2643d-aa45-42fc-b63e-e0f75df4c049 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:52.879549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:52.902627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed818d-dbbb-41bd-a0db-be909dfbe82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:52.903759 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed818d-dbbb-41bd-a0db-be909dfbe82e None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:52.904035 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed818d-dbbb-41bd-a0db-be909dfbe82e None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:52.904305 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed818d-dbbb-41bd-a0db-be909dfbe82e None admin] PATH_INFO: `/v3/projects/f39c31704e364effbc1f39d76b609302` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:52.905476 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed818d-dbbb-41bd-a0db-be909dfbe82e None admin] RBAC: Authorizing `identity:get_project(project_id=f39c31704e364effbc1f39d76b609302)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:52.906930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed818d-dbbb-41bd-a0db-be909dfbe82e None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:52.912820 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 221/442] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:52 2022] GET /identity/v3/projects/f39c31704e364effbc1f39d76b609302 => generated 302 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:52.912820 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed818d-dbbb-41bd-a0db-be909dfbe82e None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:52.992422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2643d-aa45-42fc-b63e-e0f75df4c049 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:52.998636 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:53.008544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed818d-dbbb-41bd-a0db-be909dfbe82e None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:53.027969 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc783dcf-ebb4-4b95-ae94-ca95c81dfdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:53.028885 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc783dcf-ebb4-4b95-ae94-ca95c81dfdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:53.029177 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc783dcf-ebb4-4b95-ae94-ca95c81dfdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:53.029459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc783dcf-ebb4-4b95-ae94-ca95c81dfdd0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:53.030278 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc783dcf-ebb4-4b95-ae94-ca95c81dfdd0 None admin] RBAC: Adding query filter params (group.id=2fc0a3a06f37449d8ab42381c30ef043, role.id=bff4f694fb57451e9a0783dc014187f2, scope.project.id=9bdf56f0db064649a642e4d566966738) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:53.030616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc783dcf-ebb4-4b95-ae94-ca95c81dfdd0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:53.032080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc783dcf-ebb4-4b95-ae94-ca95c81dfdd0 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:53.037779 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 222/443] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 22:47:52 2022] GET /identity/v3/role_assignments?group.id=2fc0a3a06f37449d8ab42381c30ef043&scope.project.id=9bdf56f0db064649a642e4d566966738&role.id=bff4f694fb57451e9a0783dc014187f2 => generated 611 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:53.038537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc783dcf-ebb4-4b95-ae94-ca95c81dfdd0 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:53.041449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3fdc4-3819-4cb8-80fc-ef17d49a38fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:53.042497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3fdc4-3819-4cb8-80fc-ef17d49a38fe None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:53.042497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3fdc4-3819-4cb8-80fc-ef17d49a38fe None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:53.042809 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae3fdc4-3819-4cb8-80fc-ef17d49a38fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:53.111448 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:53.124847 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3fdc4-3819-4cb8-80fc-ef17d49a38fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:53.126667 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae3fdc4-3819-4cb8-80fc-ef17d49a38fe None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:53.150074 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc783dcf-ebb4-4b95-ae94-ca95c81dfdd0 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:53.187831 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22b7f0-96d3-46b8-9292-37049158f853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:53.188640 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22b7f0-96d3-46b8-9292-37049158f853 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:53.188927 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22b7f0-96d3-46b8-9292-37049158f853 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:53.189203 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b22b7f0-96d3-46b8-9292-37049158f853 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:53.200754 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b22b7f0-96d3-46b8-9292-37049158f853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:53.209308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-1b22b7f0-96d3-46b8-9292-37049158f853 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:47:53.209308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:53.209308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:53.209308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:53.209308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:53.209308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:53.209308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:53.209308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:53.209308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:53.209308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:53.209308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:53.209308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:53.209308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:53.209308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:53.209308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:53.209308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:53.210813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:53.210813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:53.210813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:53.210813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:53.210813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:53.210813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:53.210813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:53.210813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:53.210813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:53.210813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:53.210813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:53.210813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:53.210813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:53.210813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:53.210813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:53.210813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:53.212817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:53.212817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:53.212817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:53.212817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:53.212817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:53.212817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:53.212817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:53.212817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:53.212817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:53.212817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:53.212817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:47:53.212817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:53.214939 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 223/444] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:47:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:53.215809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22b7f0-96d3-46b8-9292-37049158f853 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:53.239073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:53.254766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 222/445] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:53.254766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3fdc4-3819-4cb8-80fc-ef17d49a38fe None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:53.346493 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b22b7f0-96d3-46b8-9292-37049158f853 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:53.376086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:53.381670 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae3fdc4-3819-4cb8-80fc-ef17d49a38fe None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:53.387489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb83ff7-45e4-491b-b437-1ccba3b7724a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:53.388362 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb83ff7-45e4-491b-b437-1ccba3b7724a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:53.388648 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb83ff7-45e4-491b-b437-1ccba3b7724a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:53.388930 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb83ff7-45e4-491b-b437-1ccba3b7724a None admin] PATH_INFO: `/v3/groups/2fc0a3a06f37449d8ab42381c30ef043` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:53.390132 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb83ff7-45e4-491b-b437-1ccba3b7724a None admin] RBAC: Authorizing `identity:get_group(group_id=2fc0a3a06f37449d8ab42381c30ef043)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:53.391807 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb83ff7-45e4-491b-b437-1ccba3b7724a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:53.394638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 224/446] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 22:47:53 2022] GET /identity/v3/groups/2fc0a3a06f37449d8ab42381c30ef043 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:53.395454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb83ff7-45e4-491b-b437-1ccba3b7724a None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:53.423483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fe759-36e9-47db-88aa-c1fcbef02d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:53.424162 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fe759-36e9-47db-88aa-c1fcbef02d39 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:53.424451 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fe759-36e9-47db-88aa-c1fcbef02d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:53.424736 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137fe759-36e9-47db-88aa-c1fcbef02d39 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:53.425484 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137fe759-36e9-47db-88aa-c1fcbef02d39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:53.425797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137fe759-36e9-47db-88aa-c1fcbef02d39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:53.427209 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137fe759-36e9-47db-88aa-c1fcbef02d39 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:53.436098 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 223/447] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:47:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:53.436805 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fe759-36e9-47db-88aa-c1fcbef02d39 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:53.486870 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:53.500253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb83ff7-45e4-491b-b437-1ccba3b7724a None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:53.532916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008e95e-133e-4d84-a24d-9b2e82b4bcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:53.533755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137fe759-36e9-47db-88aa-c1fcbef02d39 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:53.533893 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008e95e-133e-4d84-a24d-9b2e82b4bcd3 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:53.534044 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008e95e-133e-4d84-a24d-9b2e82b4bcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:53.534553 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0008e95e-133e-4d84-a24d-9b2e82b4bcd3 None admin] PATH_INFO: `/v3/projects/9bdf56f0db064649a642e4d566966738` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:53.535533 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008e95e-133e-4d84-a24d-9b2e82b4bcd3 None admin] RBAC: Authorizing `identity:get_project(project_id=9bdf56f0db064649a642e4d566966738)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:53.536961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0008e95e-133e-4d84-a24d-9b2e82b4bcd3 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:53.539176 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 225/448] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:53 2022] GET /identity/v3/projects/9bdf56f0db064649a642e4d566966738 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:53.539938 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008e95e-133e-4d84-a24d-9b2e82b4bcd3 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:53.571156 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f18c6-4d3a-4900-86d5-7a55f8eb72ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:53.571812 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f18c6-4d3a-4900-86d5-7a55f8eb72ec None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:53.572095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f18c6-4d3a-4900-86d5-7a55f8eb72ec None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:53.572373 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f18c6-4d3a-4900-86d5-7a55f8eb72ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:53.573194 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f18c6-4d3a-4900-86d5-7a55f8eb72ec None admin] RBAC: Adding query filter params (group.id=2fc0a3a06f37449d8ab42381c30ef043, role.id=bff4f694fb57451e9a0783dc014187f2, scope.project.id=f39c31704e364effbc1f39d76b609302) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:53.573503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f18c6-4d3a-4900-86d5-7a55f8eb72ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:53.574957 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f18c6-4d3a-4900-86d5-7a55f8eb72ec None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:53.580935 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 224/449] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 22:47:53 2022] GET /identity/v3/role_assignments?group.id=2fc0a3a06f37449d8ab42381c30ef043&scope.project.id=f39c31704e364effbc1f39d76b609302&role.id=bff4f694fb57451e9a0783dc014187f2 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:53.581576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f18c6-4d3a-4900-86d5-7a55f8eb72ec None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:53.617289 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:53.661910 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0008e95e-133e-4d84-a24d-9b2e82b4bcd3 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:53.698131 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f18c6-4d3a-4900-86d5-7a55f8eb72ec None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:53.700045 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bf0bd-1f71-452d-8f0f-ae51d5f580b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:53.700716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bf0bd-1f71-452d-8f0f-ae51d5f580b9 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:53.701000 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bf0bd-1f71-452d-8f0f-ae51d5f580b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:53.701282 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4bf0bd-1f71-452d-8f0f-ae51d5f580b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:53.716864 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:53.737925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6678222a-62d5-44b6-8562-fd13e134af20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:53.738881 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678222a-62d5-44b6-8562-fd13e134af20 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:53.739170 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678222a-62d5-44b6-8562-fd13e134af20 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:53.739481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6678222a-62d5-44b6-8562-fd13e134af20 None admin] PATH_INFO: `/v3/groups/2fc0a3a06f37449d8ab42381c30ef043` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:53.741196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6678222a-62d5-44b6-8562-fd13e134af20 None admin] RBAC: Authorizing `identity:get_group(group_id=2fc0a3a06f37449d8ab42381c30ef043)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:53.743561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6678222a-62d5-44b6-8562-fd13e134af20 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:53.745413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 225/450] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 22:47:53 2022] GET /identity/v3/groups/2fc0a3a06f37449d8ab42381c30ef043 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:53.752361 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6678222a-62d5-44b6-8562-fd13e134af20 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:53.780365 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bf0bd-1f71-452d-8f0f-ae51d5f580b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:53.781820 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4bf0bd-1f71-452d-8f0f-ae51d5f580b9 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:53.846790 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:53.858885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6678222a-62d5-44b6-8562-fd13e134af20 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:53.886654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 226/451] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:47:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:47:53.892166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bf0bd-1f71-452d-8f0f-ae51d5f580b9 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:53.895347 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f7128-bf40-4402-8db7-6bf963580ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:53.896005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f7128-bf40-4402-8db7-6bf963580ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:53.896284 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f7128-bf40-4402-8db7-6bf963580ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:53.896557 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92f7128-bf40-4402-8db7-6bf963580ce1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:53.898031 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f7128-bf40-4402-8db7-6bf963580ce1 None admin] RBAC: Adding query filter params (group.id=2fc0a3a06f37449d8ab42381c30ef043, role.id=bda1c48ca2534bc6b2ca4f3ff479b143, scope.project.id=9bdf56f0db064649a642e4d566966738) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:53.898345 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f7128-bf40-4402-8db7-6bf963580ce1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:53.899818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92f7128-bf40-4402-8db7-6bf963580ce1 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:53.914816 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 226/452] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 22:47:53 2022] GET /identity/v3/role_assignments?group.id=2fc0a3a06f37449d8ab42381c30ef043&scope.project.id=9bdf56f0db064649a642e4d566966738&role.id=bda1c48ca2534bc6b2ca4f3ff479b143 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:47:53.998720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4bf0bd-1f71-452d-8f0f-ae51d5f580b9 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:54.034491 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f612771-a30c-4fb9-bf64-2e79d2a7e572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:54.035385 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f612771-a30c-4fb9-bf64-2e79d2a7e572 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:54.035667 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f612771-a30c-4fb9-bf64-2e79d2a7e572 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:54.035942 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f612771-a30c-4fb9-bf64-2e79d2a7e572 None admin] PATH_INFO: `/v3/projects/2ffbb7bcd828426c857e7fce279519f7` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:54.037145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f612771-a30c-4fb9-bf64-2e79d2a7e572 None admin] RBAC: Authorizing `identity:get_project(project_id=2ffbb7bcd828426c857e7fce279519f7)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:54.039201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f612771-a30c-4fb9-bf64-2e79d2a7e572 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:54.040941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 227/453] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:47:53 2022] GET /identity/v3/projects/2ffbb7bcd828426c857e7fce279519f7 => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:54.046536 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f7128-bf40-4402-8db7-6bf963580ce1 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:54.095913 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:54.148286 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92f7128-bf40-4402-8db7-6bf963580ce1 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:54.181961 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb39192c-b731-4ae0-bcf8-8d25bc4b4129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:54.182768 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39192c-b731-4ae0-bcf8-8d25bc4b4129 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:54.183053 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39192c-b731-4ae0-bcf8-8d25bc4b4129 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:54.183326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb39192c-b731-4ae0-bcf8-8d25bc4b4129 None admin] PATH_INFO: `/v3/groups/0dc79dd1f4674c7face18abdc242b9d4` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:54.184473 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb39192c-b731-4ae0-bcf8-8d25bc4b4129 None admin] RBAC: Authorizing `identity:get_group(group_id=0dc79dd1f4674c7face18abdc242b9d4)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:54.185929 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb39192c-b731-4ae0-bcf8-8d25bc4b4129 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:54.187572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 227/454] 10.222.0.17 () {56 vars in 1180 bytes} [Thu Jun 16 22:47:54 2022] GET /identity/v3/groups/0dc79dd1f4674c7face18abdc242b9d4 => generated 229 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:54.193500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f612771-a30c-4fb9-bf64-2e79d2a7e572 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:54.224865 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:54.296831 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f612771-a30c-4fb9-bf64-2e79d2a7e572 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:54.330235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8362488a-7068-466e-a458-c37a2ca53e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:54.330923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362488a-7068-466e-a458-c37a2ca53e8a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:54.331205 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362488a-7068-466e-a458-c37a2ca53e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:54.331480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8362488a-7068-466e-a458-c37a2ca53e8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:54.332301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362488a-7068-466e-a458-c37a2ca53e8a None admin] RBAC: Adding query filter params (group.id=0dc79dd1f4674c7face18abdc242b9d4, role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.project.id=2ffbb7bcd828426c857e7fce279519f7) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:54.332616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362488a-7068-466e-a458-c37a2ca53e8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:54.334036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8362488a-7068-466e-a458-c37a2ca53e8a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:54.340061 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 228/455] 10.222.0.17 () {56 vars in 1376 bytes} [Thu Jun 16 22:47:54 2022] GET /identity/v3/role_assignments?group.id=0dc79dd1f4674c7face18abdc242b9d4&scope.project.id=2ffbb7bcd828426c857e7fce279519f7&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:56.832351 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ae712-1e62-4803-b20d-1fe4a05a75a4 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:56.832867 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ae712-1e62-4803-b20d-1fe4a05a75a4 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:56.833126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ae712-1e62-4803-b20d-1fe4a05a75a4 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:56.835076 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 228/456] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:56.852056 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905e739-a781-4d76-a5ef-958f22d2cccc None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:56.852506 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905e739-a781-4d76-a5ef-958f22d2cccc None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:56.852689 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905e739-a781-4d76-a5ef-958f22d2cccc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:56.879547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-6905e739-a781-4d76-a5ef-958f22d2cccc None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:56.880730 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-6905e739-a781-4d76-a5ef-958f22d2cccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:56.970760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 229/457] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:57.007140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ae712-1e62-4803-b20d-1fe4a05a75a4 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:57.115270 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ae712-1e62-4803-b20d-1fe4a05a75a4 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:57.133202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:57.152371 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd6fc4d-c33b-43b2-a3f2-f2f3f7611ed0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:57.153174 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6fc4d-c33b-43b2-a3f2-f2f3f7611ed0 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:57.153716 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6fc4d-c33b-43b2-a3f2-f2f3f7611ed0 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:57.153992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd6fc4d-c33b-43b2-a3f2-f2f3f7611ed0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:57.165823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6fc4d-c33b-43b2-a3f2-f2f3f7611ed0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:57.167306 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd6fc4d-c33b-43b2-a3f2-f2f3f7611ed0 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:57.185167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-6bd6fc4d-c33b-43b2-a3f2-f2f3f7611ed0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:47:57.185167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:57.185167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:57.185167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:57.185167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:57.185167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:57.185167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:57.185167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:57.185167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:57.185167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:57.185167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:57.185167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:57.185167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:47:57.185167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:47:57.185167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:47:57.186077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:47:57.186077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:57.186077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:57.186077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:57.186077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:57.186077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:57.186077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:57.186077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:57.186077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:57.186077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:57.186077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:57.186077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:57.186077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:57.186077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:57.186077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:57.186077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:57.186077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:57.186953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:47:57.186953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:47:57.186953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:47:57.186953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:57.186953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 229/458] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:47:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:57.193681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905e739-a781-4d76-a5ef-958f22d2cccc None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:57.256058 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:57.257377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5299b-85cc-4a62-ace3-7f8788640ee2 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:57.257675 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5299b-85cc-4a62-ace3-7f8788640ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:57.257964 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e5299b-85cc-4a62-ace3-7f8788640ee2 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:57.260664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 230/459] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:57.264852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de944b-8dd0-44b2-ae9a-1e524ce4f9bb None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:57.265141 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de944b-8dd0-44b2-ae9a-1e524ce4f9bb None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:57.265424 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01de944b-8dd0-44b2-ae9a-1e524ce4f9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:57.297143 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-01de944b-8dd0-44b2-ae9a-1e524ce4f9bb None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:57.298338 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-01de944b-8dd0-44b2-ae9a-1e524ce4f9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:57.398528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905e739-a781-4d76-a5ef-958f22d2cccc None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:57.416390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 231/460] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:57.417904 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c97bd1-4fba-4ab4-b0d5-c2fd1f7f6c2e None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:57.418194 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c97bd1-4fba-4ab4-b0d5-c2fd1f7f6c2e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:57.418470 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c97bd1-4fba-4ab4-b0d5-c2fd1f7f6c2e None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:57.419768 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 232/461] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:57.424761 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74363d9d-8509-4ab1-ab74-84b24506ee45 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:57.425045 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74363d9d-8509-4ab1-ab74-84b24506ee45 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:57.425318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74363d9d-8509-4ab1-ab74-84b24506ee45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:57.447142 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-74363d9d-8509-4ab1-ab74-84b24506ee45 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:57.448325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-74363d9d-8509-4ab1-ab74-84b24506ee45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:57.454808 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de682f94-be59-4d96-9c29-213efbfd9ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:57.456401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de682f94-be59-4d96-9c29-213efbfd9ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:57.456896 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de682f94-be59-4d96-9c29-213efbfd9ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:57.457412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de682f94-be59-4d96-9c29-213efbfd9ff9 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:57.458836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de682f94-be59-4d96-9c29-213efbfd9ff9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:57.459888 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de682f94-be59-4d96-9c29-213efbfd9ff9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:57.462197 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de682f94-be59-4d96-9c29-213efbfd9ff9 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:57.479772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 230/462] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:57.481077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de682f94-be59-4d96-9c29-213efbfd9ff9 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:57.520911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 233/463] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:57.526637 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74363d9d-8509-4ab1-ab74-84b24506ee45 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:57.570362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:57.597008 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:57.615008 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de682f94-be59-4d96-9c29-213efbfd9ff9 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:57.659744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219b291-e461-4fff-882d-77635c92a4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:57.660967 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219b291-e461-4fff-882d-77635c92a4bf None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:57.661445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219b291-e461-4fff-882d-77635c92a4bf None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:57.661940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6219b291-e461-4fff-882d-77635c92a4bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:57.664807 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74363d9d-8509-4ab1-ab74-84b24506ee45 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:57.675769 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6219b291-e461-4fff-882d-77635c92a4bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:57.677702 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6219b291-e461-4fff-882d-77635c92a4bf None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:57.701781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679dda7-69bf-44d2-92a3-efcfe519dcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:57.702489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-6219b291-e461-4fff-882d-77635c92a4bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:47:57.702489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:57.702489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:57.702489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:57.702489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:57.702489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:57.702489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:57.702489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:57.702489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:57.702489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:57.702489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:57.702489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:57.702489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:47:57.702489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:47:57.702489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:47:57.703336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:47:57.703336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:57.703336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:57.703336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:57.703336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:57.703336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:57.703336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:57.703336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:57.703336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:57.703336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:57.703336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:57.703336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:57.703336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:57.703336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:57.703336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:57.703336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:57.703336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:57.704162 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:47:57.704162 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:47:57.704162 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:47:57.704162 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:57.704162 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 231/464] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:47:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:57.704794 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219b291-e461-4fff-882d-77635c92a4bf None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:57.705395 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679dda7-69bf-44d2-92a3-efcfe519dcc8 None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:57.705877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679dda7-69bf-44d2-92a3-efcfe519dcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:57.706380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679dda7-69bf-44d2-92a3-efcfe519dcc8 None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:57.707357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679dda7-69bf-44d2-92a3-efcfe519dcc8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:57.709960 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679dda7-69bf-44d2-92a3-efcfe519dcc8 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:57.735207 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 234/465] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 22:47:57 2022] POST /identity/v3/users => generated 246 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:47:57.736923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679dda7-69bf-44d2-92a3-efcfe519dcc8 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:57.836255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6219b291-e461-4fff-882d-77635c92a4bf None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:57.838991 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:57.861839 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679dda7-69bf-44d2-92a3-efcfe519dcc8 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:57.877110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84d4d7f-0115-4a86-8672-1367535a9007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:57.878584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d4d7f-0115-4a86-8672-1367535a9007 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:57.879082 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d4d7f-0115-4a86-8672-1367535a9007 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:57.879541 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84d4d7f-0115-4a86-8672-1367535a9007 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:57.883573 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:57.893681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d4d7f-0115-4a86-8672-1367535a9007 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:57.895480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84d4d7f-0115-4a86-8672-1367535a9007 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:57.901727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff8e757-95b8-4d75-8f8a-313c8bd1e522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:57.902662 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8e757-95b8-4d75-8f8a-313c8bd1e522 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:57.903089 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8e757-95b8-4d75-8f8a-313c8bd1e522 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:57.903775 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff8e757-95b8-4d75-8f8a-313c8bd1e522 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:57.904708 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8e757-95b8-4d75-8f8a-313c8bd1e522 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:57.905184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8e757-95b8-4d75-8f8a-313c8bd1e522 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:57.906846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff8e757-95b8-4d75-8f8a-313c8bd1e522 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:57.916051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-d84d4d7f-0115-4a86-8672-1367535a9007 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:47:57.916051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:57.916051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:57.916051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:57.916051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:57.916051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:57.916051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:57.916051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:57.916051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:57.916051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:57.916051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:57.916051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:57.916051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:47:57.916051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:47:57.916051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:47:57.917112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:47:57.917112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:57.917112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:57.917112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:57.917112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:57.917112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:57.917112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:57.917112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:57.917112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:57.917112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:57.917112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:57.917112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:57.917112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:57.917112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:57.917112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:57.917112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:57.917112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:57.918135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:47:57.918135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:47:57.918135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:47:57.918135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:57.919880 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 232/466] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:47:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:57.921530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b8b58-2727-466e-8859-db218dab07c3 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:57.921924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b8b58-2727-466e-8859-db218dab07c3 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:57.922316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0b8b58-2727-466e-8859-db218dab07c3 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:57.928260 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 233/467] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:57.929450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b8b58-2727-466e-8859-db218dab07c3 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:57.932113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 235/468] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:57.933682 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f674d-20d6-4f07-b57a-c1edf8ff70c0 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:57.934104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f674d-20d6-4f07-b57a-c1edf8ff70c0 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:57.934504 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9f674d-20d6-4f07-b57a-c1edf8ff70c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:57.954424 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-db9f674d-20d6-4f07-b57a-c1edf8ff70c0 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:57.956330 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-db9f674d-20d6-4f07-b57a-c1edf8ff70c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:58.041365 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 236/469] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:58.042394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f674d-20d6-4f07-b57a-c1edf8ff70c0 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:58.075018 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0b8b58-2727-466e-8859-db218dab07c3 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:58.116441 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f86d47-f7b6-443b-92cf-eb9f16f44ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:58.117410 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f86d47-f7b6-443b-92cf-eb9f16f44ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:58.117872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f86d47-f7b6-443b-92cf-eb9f16f44ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:58.118322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f86d47-f7b6-443b-92cf-eb9f16f44ea5 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:58.119300 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f86d47-f7b6-443b-92cf-eb9f16f44ea5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:58.119811 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f86d47-f7b6-443b-92cf-eb9f16f44ea5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:58.121547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f86d47-f7b6-443b-92cf-eb9f16f44ea5 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:58.124590 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:58.136299 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 234/470] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:47:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:58.137652 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f86d47-f7b6-443b-92cf-eb9f16f44ea5 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:58.161759 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9f674d-20d6-4f07-b57a-c1edf8ff70c0 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:58.193850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:58.210288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b5951-21ba-4b92-819e-2533f0d1f59b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:58.211023 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b5951-21ba-4b92-819e-2533f0d1f59b None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:58.211311 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b5951-21ba-4b92-819e-2533f0d1f59b None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:58.211589 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b5951-21ba-4b92-819e-2533f0d1f59b None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:58.212297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b5951-21ba-4b92-819e-2533f0d1f59b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:58.213752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b5951-21ba-4b92-819e-2533f0d1f59b None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:58.246642 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 237/471] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 22:47:58 2022] POST /identity/v3/users => generated 248 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:47:58.246642 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b5951-21ba-4b92-819e-2533f0d1f59b None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:58.266818 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f86d47-f7b6-443b-92cf-eb9f16f44ea5 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:58.307086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd7f9e-8be1-4b62-bf8b-c84e5f606554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:58.307997 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd7f9e-8be1-4b62-bf8b-c84e5f606554 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:58.308350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd7f9e-8be1-4b62-bf8b-c84e5f606554 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:58.308694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd7f9e-8be1-4b62-bf8b-c84e5f606554 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:58.320478 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd7f9e-8be1-4b62-bf8b-c84e5f606554 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:58.322174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd7f9e-8be1-4b62-bf8b-c84e5f606554 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:58.343001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-fffd7f9e-8be1-4b62-bf8b-c84e5f606554 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:47:58.343001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:58.343001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:58.343001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:58.343001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:58.343001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:58.343001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:58.343001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:58.343001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:58.343001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:58.343001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:58.343001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:58.343001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:47:58.343001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:47:58.343001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:47:58.343915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:47:58.343915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:58.343915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:58.343915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:58.343915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:58.343915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:58.343915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:58.343915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:58.343915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:58.343915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:58.343915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:58.343915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:58.343915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:58.343915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:58.343915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:58.343915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:58.343915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:58.344905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:47:58.344905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:47:58.344905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:47:58.344905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:58.344905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 235/472] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:47:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:58.346574 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72358042-7e33-4a2e-9178-21d38e4fe283 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:58.347106 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72358042-7e33-4a2e-9178-21d38e4fe283 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:58.347446 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72358042-7e33-4a2e-9178-21d38e4fe283 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:58.349389 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 236/473] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:47:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:47:58.350919 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72358042-7e33-4a2e-9178-21d38e4fe283 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:58.353671 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b5951-21ba-4b92-819e-2533f0d1f59b None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:58.384468 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:58.391550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c612113-88d0-485a-b355-036c9832c594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:58.392498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c612113-88d0-485a-b355-036c9832c594 None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:58.392971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c612113-88d0-485a-b355-036c9832c594 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:58.393362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c612113-88d0-485a-b355-036c9832c594 None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:58.394297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c612113-88d0-485a-b355-036c9832c594 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:58.396010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c612113-88d0-485a-b355-036c9832c594 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:58.427114 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 238/474] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 22:47:58 2022] POST /identity/v3/users => generated 248 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:47:58.429464 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d45e8-2d14-41bb-ab1c-77f27fd2df1a None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:58.429838 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d45e8-2d14-41bb-ab1c-77f27fd2df1a None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:58.430193 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d45e8-2d14-41bb-ab1c-77f27fd2df1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:58.461677 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-9a6d45e8-2d14-41bb-ab1c-77f27fd2df1a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:47:58.463500 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-9a6d45e8-2d14-41bb-ab1c-77f27fd2df1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:47:58.491961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:58.505347 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72358042-7e33-4a2e-9178-21d38e4fe283 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:58.546510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8bd81-ace2-4dac-a9ca-90f11f586112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:58.547253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8bd81-ace2-4dac-a9ca-90f11f586112 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:58.547541 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8bd81-ace2-4dac-a9ca-90f11f586112 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:58.547879 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b8bd81-ace2-4dac-a9ca-90f11f586112 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:58.548623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b8bd81-ace2-4dac-a9ca-90f11f586112 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:58.548929 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b8bd81-ace2-4dac-a9ca-90f11f586112 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:58.550407 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b8bd81-ace2-4dac-a9ca-90f11f586112 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:58.553811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 239/475] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:47:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:47:58.567884 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 237/476] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:47:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:47:58.573095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d45e8-2d14-41bb-ab1c-77f27fd2df1a None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:58.585617 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8bd81-ace2-4dac-a9ca-90f11f586112 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:58.689931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d45e8-2d14-41bb-ab1c-77f27fd2df1a None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:58.702011 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b8bd81-ace2-4dac-a9ca-90f11f586112 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:58.721330 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:58.735760 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92b405-1f9b-46c4-883e-7114df436ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:58.737044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92b405-1f9b-46c4-883e-7114df436ac9 None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:58.737817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92b405-1f9b-46c4-883e-7114df436ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:58.738111 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb92b405-1f9b-46c4-883e-7114df436ac9 None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:58.739215 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92b405-1f9b-46c4-883e-7114df436ac9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:58.741454 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb92b405-1f9b-46c4-883e-7114df436ac9 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:58.744377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e86d8-765b-4c27-80ae-a1cf59b63dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:58.745167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e86d8-765b-4c27-80ae-a1cf59b63dc5 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:58.745554 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e86d8-765b-4c27-80ae-a1cf59b63dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:58.745926 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3e86d8-765b-4c27-80ae-a1cf59b63dc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:58.746838 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e86d8-765b-4c27-80ae-a1cf59b63dc5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:58.748312 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3e86d8-765b-4c27-80ae-a1cf59b63dc5 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:58.757146 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.sql.core [None req-ec3e86d8-765b-4c27-80ae-a1cf59b63dc5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 22:47:58.757146 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 22:47:58.757146 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [parameters: {'id': '354a70f6ba094fe9a5223be22a5b21b8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 22:47:58.757146 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80087) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 22:47:58.765586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 240/477] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 22:47:58 2022] POST /identity/v3/users => generated 249 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:47:58.778014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-ec3e86d8-765b-4c27-80ae-a1cf59b63dc5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 22:47:58.778014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:58.778014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 22:47:58.778014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 22:47:58.778014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 22:47:58.778014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 22:47:58.778014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 22:47:58.778014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 22:47:58.778014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 22:47:58.778014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application conn.query(q) Jun 16 22:47:58.778014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 22:47:58.778014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 22:47:58.778014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 22:47:58.778014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application result.read() Jun 16 22:47:58.778014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 22:47:58.779236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 22:47:58.779236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 22:47:58.779236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 22:47:58.779236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 22:47:58.779236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 22:47:58.779236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 22:47:58.779236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 22:47:58.779236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 22:47:58.779236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:58.779236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 22:47:58.779236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:58.779236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:58.779236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 22:47:58.779236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 22:47:58.779236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 16 22:47:58.779236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return ref.to_dict() Jun 16 22:47:58.779236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 22:47:58.779236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application next(self.gen) Jun 16 22:47:58.780114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 22:47:58.780114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application yield resource Jun 16 22:47:58.780114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 22:47:58.780114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application next(self.gen) Jun 16 22:47:58.780114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 22:47:58.780114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 22:47:58.780114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 22:47:58.780114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 22:47:58.780114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 22:47:58.780114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise self.value Jun 16 22:47:58.780114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 22:47:58.780114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 22:47:58.780114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 22:47:58.780114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application session.commit() Jun 16 22:47:58.780114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 22:47:58.780114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 22:47:58.780114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 22:47:58.780975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 22:47:58.780975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 22:47:58.780975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self.session.flush() Jun 16 22:47:58.780975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 22:47:58.780975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 22:47:58.780975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 22:47:58.780975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 22:47:58.780975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 22:47:58.780975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application compat.raise_( Jun 16 22:47:58.780975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 22:47:58.780975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception Jun 16 22:47:58.780975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 22:47:58.780975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 22:47:58.780975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 22:47:58.780975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 22:47:58.780975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 22:47:58.780975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 22:47:58.781839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 22:47:58.781839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 22:47:58.781839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 22:47:58.781839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 22:47:58.781839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 22:47:58.781839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 22:47:58.781839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 22:47:58.781839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 22:47:58.781839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 22:47:58.781839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 22:47:58.781839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 22:47:58.781839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 22:47:58.781839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 22:47:58.781839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 22:47:58.781839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 22:47:58.781839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception Jun 16 22:47:58.782694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 22:47:58.782694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 22:47:58.782694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 22:47:58.782694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 22:47:58.782694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 22:47:58.782694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 22:47:58.782694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 22:47:58.782694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application conn.query(q) Jun 16 22:47:58.782694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 22:47:58.782694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 22:47:58.782694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 22:47:58.782694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application result.read() Jun 16 22:47:58.782694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 22:47:58.782694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 22:47:58.782694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 22:47:58.782694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 22:47:58.783461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 22:47:58.783461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 22:47:58.783461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 22:47:58.783461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 22:47:58.783461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 22:47:58.783461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 22:47:58.783461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [parameters: {'id': '354a70f6ba094fe9a5223be22a5b21b8', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 22:47:58.783461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 22:47:58.783461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:58.783461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:47:58.783461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:58.783461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:58.783461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:58.783461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:58.783461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:58.783461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:58.785755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:58.785755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:58.785755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:58.785755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:58.785755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:58.785755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:58.785755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 16 22:47:58.785755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 16 22:47:58.785755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:58.785755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:58.785755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 16 22:47:58.785755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 16 22:47:58.785755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 22:47:58.785755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 22:47:58.785755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 22:47:58.785755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:47:58.786759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 238/478] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jun 16 22:47:58 2022] POST /identity/v3/roles => generated 144 bytes in 195 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:47:58.786835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92b405-1f9b-46c4-883e-7114df436ac9 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:58.902746 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb92b405-1f9b-46c4-883e-7114df436ac9 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:58.940027 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915a620c-970c-4deb-b1bc-9e2f5737cb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:58.940860 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a620c-970c-4deb-b1bc-9e2f5737cb06 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:58.941146 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a620c-970c-4deb-b1bc-9e2f5737cb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:58.941423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915a620c-970c-4deb-b1bc-9e2f5737cb06 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:58.954046 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915a620c-970c-4deb-b1bc-9e2f5737cb06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:47:58.962765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-915a620c-970c-4deb-b1bc-9e2f5737cb06 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:58.962765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:47:58.962765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:47:58.962765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:47:58.962765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:47:58.962765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:47:58.962765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:47:58.962765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:47:58.962765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:47:58.962765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:47:58.962765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:47:58.962765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:47:58.962765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:47:58.962765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:47:58.962765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:47:58.962765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:47:58.964813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:47:58.964813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:47:58.964813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:47:58.964813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:47:58.964813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:47:58.964813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:47:58.964813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:47:58.964813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:47:58.964813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:47:58.964813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:47:58.964813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:47:58.964813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:47:58.964813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:47:58.964813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:47:58.964813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:47:58.964813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:47:58.966795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:47:58.966795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:47:58.966795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:47:58.966795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:47:58.966795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:47:58.966795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:47:58.966795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:47:58.966795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:47:58.966795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:47:58.966795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:47:58.966795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 22:47:58.966795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:47:58.966795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 241/479] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:47:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:47:58.970073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e86d8-765b-4c27-80ae-a1cf59b63dc5 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:47:58.999023 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:47:59.079989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3e86d8-765b-4c27-80ae-a1cf59b63dc5 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:47:59.120572 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61efb29-7ff4-45f5-810d-3baeedb03050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:47:59.121361 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61efb29-7ff4-45f5-810d-3baeedb03050 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:47:59.121650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61efb29-7ff4-45f5-810d-3baeedb03050 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:47:59.121925 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61efb29-7ff4-45f5-810d-3baeedb03050 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:47:59.122701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61efb29-7ff4-45f5-810d-3baeedb03050 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:47:59.123014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61efb29-7ff4-45f5-810d-3baeedb03050 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:47:59.124379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61efb29-7ff4-45f5-810d-3baeedb03050 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:47:59.134424 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 239/480] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:47:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:00.329104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d450b8-2857-4d16-afd1-6e64280412b9 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:00.330093 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d450b8-2857-4d16-afd1-6e64280412b9 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:00.330572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d450b8-2857-4d16-afd1-6e64280412b9 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:00.332597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 242/481] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:00.339359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02dab8b-9050-4923-9979-1abc76c93cb4 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:00.340280 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02dab8b-9050-4923-9979-1abc76c93cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:00.340755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02dab8b-9050-4923-9979-1abc76c93cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:00.358249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-d02dab8b-9050-4923-9979-1abc76c93cb4 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:00.359737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-d02dab8b-9050-4923-9979-1abc76c93cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:00.430962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 240/482] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:00.464838 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d450b8-2857-4d16-afd1-6e64280412b9 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:00.522269 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:00.615058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d450b8-2857-4d16-afd1-6e64280412b9 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:00.665639 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff1e81-e313-4205-b56d-faa5014d70e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:00.666839 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff1e81-e313-4205-b56d-faa5014d70e4 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:00.667341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff1e81-e313-4205-b56d-faa5014d70e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:00.667831 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff1e81-e313-4205-b56d-faa5014d70e4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:00.687476 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff1e81-e313-4205-b56d-faa5014d70e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:00.695189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-00ff1e81-e313-4205-b56d-faa5014d70e4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:00.695189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:00.695189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:00.695189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:00.695189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:00.695189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:00.695189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:00.695189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:00.695189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:00.695189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:00.695189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:00.695189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:00.695189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:00.695189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:00.695189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:00.695189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:00.696075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:00.696075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:00.696075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:00.696075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:00.696075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:00.696075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:00.696075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:00.696075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:00.696075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:00.696075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:00.696075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:00.696075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:00.696075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:00.696075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:00.696075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:00.696075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:00.697274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:00.697274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:00.697274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:00.697274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:00.697274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:00.697274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:00.697274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:00.697274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:00.697274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:00.697274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:00.697274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:00.697274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:00.697274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 243/483] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:00.707686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02dab8b-9050-4923-9979-1abc76c93cb4 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:00.734898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:00.886795 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02dab8b-9050-4923-9979-1abc76c93cb4 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:00.938403 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa9edc-72e4-43f3-8ce0-447d60dae660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:00.939122 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa9edc-72e4-43f3-8ce0-447d60dae660 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:00.939406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa9edc-72e4-43f3-8ce0-447d60dae660 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:00.939686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa9edc-72e4-43f3-8ce0-447d60dae660 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:00.940431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa9edc-72e4-43f3-8ce0-447d60dae660 None admin] RBAC: Adding query filter params (name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:00.940742 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa9edc-72e4-43f3-8ce0-447d60dae660 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:00.942197 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfa9edc-72e4-43f3-8ce0-447d60dae660 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:00.952298 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 241/484] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:48:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:00.963277 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff1e81-e313-4205-b56d-faa5014d70e4 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:01.107920 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:01.115678 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348acd0-ccc4-4b7e-a6ca-3d6fdf78fc24 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:01.116063 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348acd0-ccc4-4b7e-a6ca-3d6fdf78fc24 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:01.116410 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f348acd0-ccc4-4b7e-a6ca-3d6fdf78fc24 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:01.121489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 242/485] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:01.124412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53929cb-ea37-4c10-9c18-4ae4cb7ae649 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:01.125142 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53929cb-ea37-4c10-9c18-4ae4cb7ae649 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:01.125498 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53929cb-ea37-4c10-9c18-4ae4cb7ae649 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:01.133648 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff1e81-e313-4205-b56d-faa5014d70e4 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:01.146976 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-e53929cb-ea37-4c10-9c18-4ae4cb7ae649 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:01.148264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e53929cb-ea37-4c10-9c18-4ae4cb7ae649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:01.205852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56589c64-cdfb-4f75-9190-7ac58191dc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:01.206897 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56589c64-cdfb-4f75-9190-7ac58191dc97 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:01.207180 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56589c64-cdfb-4f75-9190-7ac58191dc97 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:01.207549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56589c64-cdfb-4f75-9190-7ac58191dc97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:01.220490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56589c64-cdfb-4f75-9190-7ac58191dc97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:01.222227 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56589c64-cdfb-4f75-9190-7ac58191dc97 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:01.241313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 243/486] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:01.242324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-56589c64-cdfb-4f75-9190-7ac58191dc97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:01.242324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:01.242324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:01.242324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:01.242324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:01.242324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:01.242324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:01.242324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:01.242324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:01.242324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:01.242324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:01.242324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:01.242324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:01.242324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:01.242324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:01.243172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:01.243172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:01.243172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:01.243172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:01.243172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:01.243172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:01.243172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:01.243172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:01.243172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:01.243172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:01.243172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:01.243172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:01.243172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:01.243172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:01.243172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:01.243172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:01.243172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:01.244003 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:01.244003 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:01.244003 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:01.244003 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:01.244206 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92a6a3-ce31-47ef-be52-78411f40d6ce None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:01.244206 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92a6a3-ce31-47ef-be52-78411f40d6ce None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:01.244206 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e92a6a3-ce31-47ef-be52-78411f40d6ce None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:01.244381 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 244/487] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:01.244873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 244/488] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:01.249756 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57782ea7-3776-41ff-bde6-cf9e4a435882 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:01.250100 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57782ea7-3776-41ff-bde6-cf9e4a435882 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:01.250386 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57782ea7-3776-41ff-bde6-cf9e4a435882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:01.259725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92a6a3-ce31-47ef-be52-78411f40d6ce None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:01.274615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-57782ea7-3776-41ff-bde6-cf9e4a435882 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:01.274615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-57782ea7-3776-41ff-bde6-cf9e4a435882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:01.274857 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:01.348175 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 245/489] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:48:01.349023 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57782ea7-3776-41ff-bde6-cf9e4a435882 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:01.375349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e92a6a3-ce31-47ef-be52-78411f40d6ce None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:01.415378 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b8d31-2d13-4979-8c70-6b8c253d7172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:01.415962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b8d31-2d13-4979-8c70-6b8c253d7172 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:01.416248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b8d31-2d13-4979-8c70-6b8c253d7172 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:01.416524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77b8d31-2d13-4979-8c70-6b8c253d7172 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:01.417408 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b8d31-2d13-4979-8c70-6b8c253d7172 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:01.417721 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b8d31-2d13-4979-8c70-6b8c253d7172 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:01.419244 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77b8d31-2d13-4979-8c70-6b8c253d7172 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:01.432979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 245/490] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:01.433881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b8d31-2d13-4979-8c70-6b8c253d7172 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:01.478699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:01.516403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57782ea7-3776-41ff-bde6-cf9e4a435882 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:01.525186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:01.553635 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86f53f-6a63-4c48-a639-f9a8d8e146ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:01.554443 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86f53f-6a63-4c48-a639-f9a8d8e146ca None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:01.554765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86f53f-6a63-4c48-a639-f9a8d8e146ca None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:01.555046 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a86f53f-6a63-4c48-a639-f9a8d8e146ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:01.563029 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77b8d31-2d13-4979-8c70-6b8c253d7172 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:01.567829 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a86f53f-6a63-4c48-a639-f9a8d8e146ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:01.577585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-8a86f53f-6a63-4c48-a639-f9a8d8e146ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:01.577585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:01.577585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:01.577585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:01.577585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:01.577585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:01.577585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:01.577585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:01.577585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:01.577585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:01.577585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:01.577585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:01.577585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:01.577585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:01.577585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:01.577585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:01.578486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:01.578486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:01.578486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:01.578486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:01.578486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:01.578486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:01.578486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:01.578486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:01.578486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:01.578486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:01.578486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:01.578486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:01.578486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:01.578486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:01.578486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:01.578486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:01.579614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:01.579614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:01.579614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:01.579614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:01.579614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:01.579614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:01.579614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:01.579614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:01.579614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:01.579614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:01.579614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:01.579614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:01.580735 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 246/491] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:01.581598 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86f53f-6a63-4c48-a639-f9a8d8e146ca None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:01.602251 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f167b8ff-bb1c-43a8-8747-3e15ece63e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:01.603486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167b8ff-bb1c-43a8-8747-3e15ece63e8d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:01.604001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167b8ff-bb1c-43a8-8747-3e15ece63e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:01.604509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167b8ff-bb1c-43a8-8747-3e15ece63e8d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:01.615075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f167b8ff-bb1c-43a8-8747-3e15ece63e8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:01.623322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-f167b8ff-bb1c-43a8-8747-3e15ece63e8d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:01.623322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:01.623322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:01.623322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:01.623322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:01.623322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:01.623322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:01.623322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:01.623322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:01.623322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:01.623322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:01.623322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:01.623322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:01.623322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:01.623322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:01.623322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:01.624218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:01.624218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:01.624218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:01.624218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:01.624218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:01.624218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:01.624218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:01.624218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:01.624218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:01.624218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:01.624218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:01.624218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:01.624218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:01.624218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:01.624218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:01.624218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:01.624990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:01.624990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:01.624990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:01.624990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:01.624990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:01.624990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:01.624990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:01.624990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:01.624990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:01.624990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:01.624990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:01.624990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:01.629228 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 246/492] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:01.630529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d88519-52a7-4cfc-8e8a-13974e392008 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:01.631110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d88519-52a7-4cfc-8e8a-13974e392008 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:01.631228 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d88519-52a7-4cfc-8e8a-13974e392008 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:01.632556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 247/493] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:01.633078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d88519-52a7-4cfc-8e8a-13974e392008 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:01.696465 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a86f53f-6a63-4c48-a639-f9a8d8e146ca None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:01.733914 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9499ec-864c-4fa5-a4c7-2dae53156125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:01.734750 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9499ec-864c-4fa5-a4c7-2dae53156125 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:01.735044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9499ec-864c-4fa5-a4c7-2dae53156125 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:01.735464 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de9499ec-864c-4fa5-a4c7-2dae53156125 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:01.737478 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:01.739215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d88519-52a7-4cfc-8e8a-13974e392008 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:01.745205 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9499ec-864c-4fa5-a4c7-2dae53156125 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:01.747146 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de9499ec-864c-4fa5-a4c7-2dae53156125 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:01.763660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-de9499ec-864c-4fa5-a4c7-2dae53156125 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 22:48:01.763660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:01.763660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:01.763660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:01.763660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:01.763660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:01.763660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:01.763660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:01.763660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:01.763660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:01.763660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:01.763660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:01.763660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:01.763660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:01.763660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:01.763660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:01.764743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:01.764743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:01.764743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:01.764743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:01.764743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:01.764743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:01.764743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:01.764743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:01.764743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:01.764743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:01.764743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:01.764743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:01.764743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:01.764743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:01.764743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:01.764743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:01.764743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:01.764743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:01.765732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:01.765732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:01.765732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:01.765732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:01.765732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:01.765732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:01.765732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:01.765732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:01.765732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 22:48:01.765732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:01.765732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 247/494] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jun 16 22:48:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:01.766296 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9499ec-864c-4fa5-a4c7-2dae53156125 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:01.774089 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:01.785297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7735cd87-2df6-42bf-9531-8d4db9dc98cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:01.786493 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735cd87-2df6-42bf-9531-8d4db9dc98cc None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:01.786832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735cd87-2df6-42bf-9531-8d4db9dc98cc None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:01.787115 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7735cd87-2df6-42bf-9531-8d4db9dc98cc None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:01.787958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735cd87-2df6-42bf-9531-8d4db9dc98cc None admin] RBAC: Adding query filter params (name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:01.789754 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735cd87-2df6-42bf-9531-8d4db9dc98cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:01.791792 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7735cd87-2df6-42bf-9531-8d4db9dc98cc None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:01.800999 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 248/495] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:48:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:01.806725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd1cea-fbb3-47fc-a874-49d01b01aabd None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:01.807027 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd1cea-fbb3-47fc-a874-49d01b01aabd None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:01.807312 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd1cea-fbb3-47fc-a874-49d01b01aabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:01.823856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-60bd1cea-fbb3-47fc-a874-49d01b01aabd None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:01.825041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-60bd1cea-fbb3-47fc-a874-49d01b01aabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:01.874538 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de9499ec-864c-4fa5-a4c7-2dae53156125 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:01.896057 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 249/496] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:01.907545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd1cea-fbb3-47fc-a874-49d01b01aabd None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:01.912254 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a5f8b-8cbc-43d2-8d48-0b23ee2218ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:01.913212 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a5f8b-8cbc-43d2-8d48-0b23ee2218ad None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:01.913553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a5f8b-8cbc-43d2-8d48-0b23ee2218ad None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:01.913883 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1a5f8b-8cbc-43d2-8d48-0b23ee2218ad None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:01.914654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a5f8b-8cbc-43d2-8d48-0b23ee2218ad None admin] RBAC: Adding query filter params (name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:01.914968 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a5f8b-8cbc-43d2-8d48-0b23ee2218ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:01.916414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c1a5f8b-8cbc-43d2-8d48-0b23ee2218ad None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:01.931446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 248/497] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:48:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:01.931446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a5f8b-8cbc-43d2-8d48-0b23ee2218ad None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:01.994875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:02.019672 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd1cea-fbb3-47fc-a874-49d01b01aabd None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:02.040784 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:02.040934 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1a5f8b-8cbc-43d2-8d48-0b23ee2218ad None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:02.056218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b63a91-7bf9-40da-beab-d31062522d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:02.056884 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b63a91-7bf9-40da-beab-d31062522d4b None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:02.057172 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b63a91-7bf9-40da-beab-d31062522d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:02.057449 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b63a91-7bf9-40da-beab-d31062522d4b None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:02.058280 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b63a91-7bf9-40da-beab-d31062522d4b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:02.058618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b63a91-7bf9-40da-beab-d31062522d4b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:02.060036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b63a91-7bf9-40da-beab-d31062522d4b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:02.076456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f4911-6242-43c4-a854-b10584d13467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:02.077973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f4911-6242-43c4-a854-b10584d13467 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:02.078345 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f4911-6242-43c4-a854-b10584d13467 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:02.078551 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667f4911-6242-43c4-a854-b10584d13467 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:02.089029 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f4911-6242-43c4-a854-b10584d13467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:02.090501 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667f4911-6242-43c4-a854-b10584d13467 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:02.096756 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 250/498] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jun 16 22:48:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:02.097568 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b63a91-7bf9-40da-beab-d31062522d4b None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:02.107627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-667f4911-6242-43c4-a854-b10584d13467 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:02.107627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:02.107627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:02.107627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:02.107627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:02.107627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:02.107627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:02.107627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:02.107627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:02.107627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:02.107627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:02.107627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:02.107627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:02.107627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:02.107627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:02.108724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:02.108724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:02.108724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:02.108724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:02.108724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:02.108724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:02.108724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:02.108724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:02.108724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:02.108724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:02.108724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:02.108724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:02.108724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:02.108724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:02.108724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:02.108724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:02.108724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:02.109554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:02.109554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:02.109554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:02.109554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:02.109554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 249/499] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:02.109554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f4911-6242-43c4-a854-b10584d13467 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:02.200879 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b63a91-7bf9-40da-beab-d31062522d4b None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:02.207964 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667f4911-6242-43c4-a854-b10584d13467 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:02.216809 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:02.235758 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cdc42c-2697-43ed-a3ee-89d21973aa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:02.236544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdc42c-2697-43ed-a3ee-89d21973aa79 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:02.236825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdc42c-2697-43ed-a3ee-89d21973aa79 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:02.237098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cdc42c-2697-43ed-a3ee-89d21973aa79 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:02.243459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771835a2-5953-4982-bf2a-4defac5c1078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:02.244254 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771835a2-5953-4982-bf2a-4defac5c1078 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:02.244541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771835a2-5953-4982-bf2a-4defac5c1078 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:02.244819 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771835a2-5953-4982-bf2a-4defac5c1078 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:02.247467 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cdc42c-2697-43ed-a3ee-89d21973aa79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:02.255003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-94cdc42c-2697-43ed-a3ee-89d21973aa79 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:02.255003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:02.255003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:02.255003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:02.255003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:02.255003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:02.255003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:02.255003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:02.255003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:02.255003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:02.255003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:02.255003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:02.255003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:02.255003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:02.255003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:02.255003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:02.255873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:02.255873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:02.255873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:02.255873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:02.255873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:02.255873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:02.255873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:02.255873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:02.255873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:02.255873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:02.255873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:02.255873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:02.255873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:02.255873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:02.255873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:02.255873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:02.257269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:02.257269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:02.257269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:02.257269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:02.257269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:02.257269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:02.257269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:02.257269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:02.257269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:02.257269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:02.257269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:02.257269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:02.258744 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771835a2-5953-4982-bf2a-4defac5c1078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:02.258889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 251/500] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:02.259011 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771835a2-5953-4982-bf2a-4defac5c1078 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:02.259131 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd509c-3e0f-4df2-8cae-48a42f9c778d None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:02.259131 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd509c-3e0f-4df2-8cae-48a42f9c778d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:02.259131 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebd509c-3e0f-4df2-8cae-48a42f9c778d None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:02.259760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 252/501] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:02 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:02.260439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd509c-3e0f-4df2-8cae-48a42f9c778d None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:02.279423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-771835a2-5953-4982-bf2a-4defac5c1078 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:02.279423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:02.279423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:02.279423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:02.279423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:02.279423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:02.279423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:02.279423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:02.279423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:02.279423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:02.279423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:02.279423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:02.279423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:02.279423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:02.279423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:02.280357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:02.280357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:02.280357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:02.280357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:02.280357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:02.280357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:02.280357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:02.280357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:02.280357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:02.280357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:02.280357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:02.280357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:02.280357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:02.280357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:02.280357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:02.280357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:02.280357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:02.281243 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:02.281243 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:02.281243 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:02.281243 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:02.281753 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 250/502] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:02.282675 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771835a2-5953-4982-bf2a-4defac5c1078 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:02.302062 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:02.363701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebd509c-3e0f-4df2-8cae-48a42f9c778d None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:02.380170 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771835a2-5953-4982-bf2a-4defac5c1078 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:02.396693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f698fb4-8cbb-401d-aeaf-cd22bfa18559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:02.397387 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f698fb4-8cbb-401d-aeaf-cd22bfa18559 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:02.397670 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f698fb4-8cbb-401d-aeaf-cd22bfa18559 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:02.397951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f698fb4-8cbb-401d-aeaf-cd22bfa18559 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:02.414440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b47d5-e672-4836-bcf4-2b02697e12ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:02.415138 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b47d5-e672-4836-bcf4-2b02697e12ad None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:02.415425 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b47d5-e672-4836-bcf4-2b02697e12ad None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:02.415706 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8b47d5-e672-4836-bcf4-2b02697e12ad None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:02.416417 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b47d5-e672-4836-bcf4-2b02697e12ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:02.416721 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b47d5-e672-4836-bcf4-2b02697e12ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:02.418098 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8b47d5-e672-4836-bcf4-2b02697e12ad None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:02.429445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 251/503] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:02.430182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b47d5-e672-4836-bcf4-2b02697e12ad None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:02.451438 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:02.466904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f698fb4-8cbb-401d-aeaf-cd22bfa18559 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:02.468365 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f698fb4-8cbb-401d-aeaf-cd22bfa18559 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:02.505656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:02.529538 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8b47d5-e672-4836-bcf4-2b02697e12ad None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:02.563657 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db32806-7612-4f66-a498-b146dc384728 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:02.564315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db32806-7612-4f66-a498-b146dc384728 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:02.564601 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db32806-7612-4f66-a498-b146dc384728 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:02.564874 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db32806-7612-4f66-a498-b146dc384728 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:02.565602 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db32806-7612-4f66-a498-b146dc384728 None admin] RBAC: Adding query filter params (name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:02.565905 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db32806-7612-4f66-a498-b146dc384728 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:02.567227 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 253/504] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:48:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:48:02.567324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db32806-7612-4f66-a498-b146dc384728 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:02.569079 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05338dcc-26d4-4605-82e0-2d4eed90904f None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:02.569374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05338dcc-26d4-4605-82e0-2d4eed90904f None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:02.569655 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05338dcc-26d4-4605-82e0-2d4eed90904f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:02.579227 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 252/505] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:48:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:02.580016 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db32806-7612-4f66-a498-b146dc384728 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:02.585694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-05338dcc-26d4-4605-82e0-2d4eed90904f None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:02.586917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-05338dcc-26d4-4605-82e0-2d4eed90904f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:02.651511 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 254/506] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:02.652311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05338dcc-26d4-4605-82e0-2d4eed90904f None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:02.661450 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:02.681091 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db32806-7612-4f66-a498-b146dc384728 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:02.713820 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508e48c-6693-4a93-8373-9d5f07fb0b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:02.714573 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508e48c-6693-4a93-8373-9d5f07fb0b95 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:02.714889 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508e48c-6693-4a93-8373-9d5f07fb0b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:02.715168 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3508e48c-6693-4a93-8373-9d5f07fb0b95 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:02.715877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508e48c-6693-4a93-8373-9d5f07fb0b95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:02.716182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508e48c-6693-4a93-8373-9d5f07fb0b95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:02.717019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:02.717681 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3508e48c-6693-4a93-8373-9d5f07fb0b95 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:02.729527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 253/507] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:02.730305 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508e48c-6693-4a93-8373-9d5f07fb0b95 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:02.749974 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05338dcc-26d4-4605-82e0-2d4eed90904f None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:02.783108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a4fc6-4b95-426d-afed-b83b22f40f50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:02.783835 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a4fc6-4b95-426d-afed-b83b22f40f50 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:02.784120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a4fc6-4b95-426d-afed-b83b22f40f50 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:02.784392 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a4fc6-4b95-426d-afed-b83b22f40f50 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:02.791597 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a4fc6-4b95-426d-afed-b83b22f40f50 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:02.793048 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a4fc6-4b95-426d-afed-b83b22f40f50 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:02.809127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-411a4fc6-4b95-426d-afed-b83b22f40f50 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 22:48:02.809127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:02.809127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:02.809127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:02.809127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:02.809127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:02.809127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:02.809127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:02.809127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:02.809127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:02.809127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:02.809127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:02.809127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:02.809127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:02.809127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:02.809127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:02.810004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:02.810004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:02.810004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:02.810004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:02.810004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:02.810004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:02.810004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:02.810004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:02.810004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:02.810004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:02.810004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:02.810004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:02.810004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:02.810004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:02.810004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:02.810004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:02.810004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:02.810004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:02.810907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:02.810907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:02.810907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:02.810907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:02.810907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:02.810907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:02.810907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:02.810907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:02.810907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 22:48:02.810907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:02.810907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 255/508] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:48:02 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:02.811471 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a4fc6-4b95-426d-afed-b83b22f40f50 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:02.835231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3508e48c-6693-4a93-8373-9d5f07fb0b95 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:02.867652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e9ef3-058c-4bd9-af5d-231f4a09807e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:02.868433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e9ef3-058c-4bd9-af5d-231f4a09807e None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:02.868716 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e9ef3-058c-4bd9-af5d-231f4a09807e None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:02.868992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8e9ef3-058c-4bd9-af5d-231f4a09807e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:02.878806 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e9ef3-058c-4bd9-af5d-231f4a09807e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:02.880252 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8e9ef3-058c-4bd9-af5d-231f4a09807e None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:02.882002 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:02.898515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-4f8e9ef3-058c-4bd9-af5d-231f4a09807e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:02.898515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:02.898515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:02.898515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:02.898515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:02.898515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:02.898515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:02.898515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:02.898515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:02.898515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:02.898515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:02.898515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:02.898515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:02.898515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:02.898515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:02.899544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:02.899544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:02.899544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:02.899544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:02.899544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:02.899544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:02.899544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:02.899544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:02.899544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:02.899544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:02.899544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:02.899544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:02.899544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:02.899544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:02.899544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:02.899544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:02.899544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:02.900546 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:02.900546 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:02.900546 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:02.900546 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:02.900546 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 254/509] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:02.900546 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e9ef3-058c-4bd9-af5d-231f4a09807e None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:02.913526 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a4fc6-4b95-426d-afed-b83b22f40f50 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:02.939838 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:02.948200 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd55a5-79c5-4d39-b1ec-0908ee298530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:02.949001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd55a5-79c5-4d39-b1ec-0908ee298530 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:02.949325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd55a5-79c5-4d39-b1ec-0908ee298530 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:02.949574 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd55a5-79c5-4d39-b1ec-0908ee298530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:02.960631 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd55a5-79c5-4d39-b1ec-0908ee298530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:02.962099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbd55a5-79c5-4d39-b1ec-0908ee298530 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:02.979141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-adbd55a5-79c5-4d39-b1ec-0908ee298530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:02.979141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:02.979141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:02.979141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:02.979141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:02.979141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:02.979141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:02.979141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:02.979141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:02.979141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:02.979141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:02.979141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:02.979141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:02.979141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:02.979141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:02.980128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:02.980128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:02.980128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:02.980128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:02.980128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:02.980128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:02.980128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:02.980128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:02.980128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:02.980128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:02.980128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:02.980128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:02.980128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:02.980128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:02.980128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:02.980128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:02.980128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:02.981114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:02.981114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:02.981114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:02.981114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:02.981114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 256/510] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:02.981447 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd55a5-79c5-4d39-b1ec-0908ee298530 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:03.007801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8e9ef3-058c-4bd9-af5d-231f4a09807e None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:03.041128 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e1d4f-b51d-475c-a8b0-12412710ba51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:03.041856 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e1d4f-b51d-475c-a8b0-12412710ba51 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:03.042157 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e1d4f-b51d-475c-a8b0-12412710ba51 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:03.042420 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e1d4f-b51d-475c-a8b0-12412710ba51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:03.052322 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e1d4f-b51d-475c-a8b0-12412710ba51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:03.054304 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1e1d4f-b51d-475c-a8b0-12412710ba51 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:03.071475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-3d1e1d4f-b51d-475c-a8b0-12412710ba51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:03.071475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:03.071475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:03.071475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:03.071475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:03.071475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:03.071475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:03.071475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:03.071475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:03.071475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:03.071475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:03.071475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:03.071475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:03.071475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:03.071475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:03.072497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:03.072497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:03.072497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:03.072497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:03.072497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:03.072497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:03.072497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:03.072497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:03.072497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:03.072497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:03.072497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:03.072497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:03.072497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:03.072497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:03.072497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:03.072497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:03.072497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:03.073449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:03.073449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:03.073449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:03.073449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:03.073449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 255/511] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:03.073774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e1d4f-b51d-475c-a8b0-12412710ba51 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:03.080459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbd55a5-79c5-4d39-b1ec-0908ee298530 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:03.115026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be49665-a8aa-4f2f-a3ee-a092f37cf2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:03.116443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be49665-a8aa-4f2f-a3ee-a092f37cf2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:03.116718 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be49665-a8aa-4f2f-a3ee-a092f37cf2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:03.116986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be49665-a8aa-4f2f-a3ee-a092f37cf2a8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:03.120541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:03.124410 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be49665-a8aa-4f2f-a3ee-a092f37cf2a8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:03.125840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be49665-a8aa-4f2f-a3ee-a092f37cf2a8 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:03.140740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-7be49665-a8aa-4f2f-a3ee-a092f37cf2a8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 22:48:03.140740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:03.140740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:03.140740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:03.140740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:03.140740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:03.140740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:03.140740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:03.140740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:03.140740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:03.140740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:03.140740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:03.140740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:03.140740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:03.140740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:03.140740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:03.141765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:03.141765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:03.141765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:03.141765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:03.141765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:03.141765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:03.141765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:03.141765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:03.141765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:03.141765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:03.141765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:03.141765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:03.141765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:03.141765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:03.141765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:03.141765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:03.141765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:03.141765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:03.142665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:03.142665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:03.142665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:03.142665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:03.142665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:03.142665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:03.142665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:03.142665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:03.142665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 22:48:03.142665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:03.142665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 257/512] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:48:02 2022] GET /identity/v3/users/cinder => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:03.143209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be49665-a8aa-4f2f-a3ee-a092f37cf2a8 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:03.176703 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e1d4f-b51d-475c-a8b0-12412710ba51 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:03.186199 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:03.215237 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1299d-bba0-499d-97e1-8523914545c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:03.215913 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1299d-bba0-499d-97e1-8523914545c6 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:03.216191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1299d-bba0-499d-97e1-8523914545c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:03.216464 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa1299d-bba0-499d-97e1-8523914545c6 None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:03.217288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1299d-bba0-499d-97e1-8523914545c6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:03.217594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1299d-bba0-499d-97e1-8523914545c6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:03.219063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa1299d-bba0-499d-97e1-8523914545c6 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:03.246201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be49665-a8aa-4f2f-a3ee-a092f37cf2a8 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:03.256654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 256/513] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jun 16 22:48:03 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:03.257463 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1299d-bba0-499d-97e1-8523914545c6 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:03.284324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961774c8-ae99-4030-b257-ac209a935081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:03.284975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961774c8-ae99-4030-b257-ac209a935081 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:03.285259 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961774c8-ae99-4030-b257-ac209a935081 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:03.285538 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961774c8-ae99-4030-b257-ac209a935081 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:03.286249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961774c8-ae99-4030-b257-ac209a935081 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:03.286556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961774c8-ae99-4030-b257-ac209a935081 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:03.288490 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961774c8-ae99-4030-b257-ac209a935081 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:03.299573 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 258/514] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:03.300298 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961774c8-ae99-4030-b257-ac209a935081 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:03.353695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa1299d-bba0-499d-97e1-8523914545c6 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:03.356186 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:03.387607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47870292-24c8-4d90-8323-f48a494b668b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:03.388273 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47870292-24c8-4d90-8323-f48a494b668b None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:03.388555 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47870292-24c8-4d90-8323-f48a494b668b None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:03.388835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47870292-24c8-4d90-8323-f48a494b668b None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:03.389535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47870292-24c8-4d90-8323-f48a494b668b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:03.389868 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47870292-24c8-4d90-8323-f48a494b668b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:03.391344 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47870292-24c8-4d90-8323-f48a494b668b None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:03.394818 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:03.397993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961774c8-ae99-4030-b257-ac209a935081 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:03.403097 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 257/515] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:03.403097 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47870292-24c8-4d90-8323-f48a494b668b None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:03.432642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44510e96-b867-41a0-b59a-83d7748de867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:03.433314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44510e96-b867-41a0-b59a-83d7748de867 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:03.433605 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44510e96-b867-41a0-b59a-83d7748de867 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:03.433886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44510e96-b867-41a0-b59a-83d7748de867 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:03.434630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44510e96-b867-41a0-b59a-83d7748de867 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:03.434951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44510e96-b867-41a0-b59a-83d7748de867 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:03.436330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44510e96-b867-41a0-b59a-83d7748de867 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:03.447814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 259/516] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:03.448591 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44510e96-b867-41a0-b59a-83d7748de867 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:03.505059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47870292-24c8-4d90-8323-f48a494b668b None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:03.537847 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533d8b6-bf52-42bd-8b8b-a534bcfa4125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:03.538773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533d8b6-bf52-42bd-8b8b-a534bcfa4125 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:03.539061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533d8b6-bf52-42bd-8b8b-a534bcfa4125 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:03.539341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533d8b6-bf52-42bd-8b8b-a534bcfa4125 None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:03.540171 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533d8b6-bf52-42bd-8b8b-a534bcfa4125 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:03.540483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533d8b6-bf52-42bd-8b8b-a534bcfa4125 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:03.541918 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533d8b6-bf52-42bd-8b8b-a534bcfa4125 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:03.546161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44510e96-b867-41a0-b59a-83d7748de867 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:03.559612 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 258/517] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jun 16 22:48:03 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:03.560332 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533d8b6-bf52-42bd-8b8b-a534bcfa4125 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:03.579983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1514aa2-dd91-4f15-a52e-ee4c22d6d1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:03.580627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1514aa2-dd91-4f15-a52e-ee4c22d6d1c9 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:03.580911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1514aa2-dd91-4f15-a52e-ee4c22d6d1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:03.581185 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1514aa2-dd91-4f15-a52e-ee4c22d6d1c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:03.583854 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:03.608394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:03.648540 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1514aa2-dd91-4f15-a52e-ee4c22d6d1c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:03.650089 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1514aa2-dd91-4f15-a52e-ee4c22d6d1c9 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:03.660129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533d8b6-bf52-42bd-8b8b-a534bcfa4125 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:03.694424 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b08ff-cec0-40cb-9466-8ecb897d0e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:03.695255 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b08ff-cec0-40cb-9466-8ecb897d0e43 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:03.695543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b08ff-cec0-40cb-9466-8ecb897d0e43 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:03.695822 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12b08ff-cec0-40cb-9466-8ecb897d0e43 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:03.702391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b08ff-cec0-40cb-9466-8ecb897d0e43 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:03.703886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12b08ff-cec0-40cb-9466-8ecb897d0e43 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:03.718303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-c12b08ff-cec0-40cb-9466-8ecb897d0e43 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:03.718303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:03.718303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:03.718303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:03.718303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:03.718303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:03.718303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:03.718303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:03.718303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:03.718303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:03.718303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:03.718303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:03.718303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:03.718303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:03.718303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:03.719360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:03.719360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:03.719360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:03.719360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:03.719360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:03.719360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:03.719360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:03.719360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:03.719360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:03.719360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:03.719360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:03.719360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:03.719360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:03.719360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:03.719360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:03.719360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:03.719360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:03.720222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:03.720222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:03.720222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:03.720222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:03.720222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:03.720222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:03.720222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:03.720222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:03.720222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 259/518] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:03.720704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b08ff-cec0-40cb-9466-8ecb897d0e43 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:03.749269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 260/519] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:48:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:48:03.750097 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1514aa2-dd91-4f15-a52e-ee4c22d6d1c9 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:03.804215 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:03.810317 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:03.822260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12b08ff-cec0-40cb-9466-8ecb897d0e43 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:03.850637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1514aa2-dd91-4f15-a52e-ee4c22d6d1c9 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:03.856571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8474f-43e6-449b-ba83-730fb9c52a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:03.857371 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8474f-43e6-449b-ba83-730fb9c52a3c None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:03.857659 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8474f-43e6-449b-ba83-730fb9c52a3c None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:03.857937 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab8474f-43e6-449b-ba83-730fb9c52a3c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:03.867121 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8474f-43e6-449b-ba83-730fb9c52a3c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:03.868584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab8474f-43e6-449b-ba83-730fb9c52a3c None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:03.882490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-9ab8474f-43e6-449b-ba83-730fb9c52a3c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 22:48:03.882490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:03.882490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:03.882490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:03.882490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:03.882490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:03.882490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:03.882490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:03.882490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:03.882490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:03.882490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:03.882490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:03.882490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:03.882490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:03.882490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:03.882490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:03.883586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:03.883586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:03.883586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:03.883586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:03.883586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:03.883586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:03.883586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:03.883586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:03.883586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:03.883586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:03.883586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:03.883586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:03.883586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:03.883586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:03.883586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:03.883586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:03.883586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:03.883586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:03.884499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:03.884499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:03.884499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:03.884499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:03.884499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:03.884499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:03.884499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:03.884499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:03.884499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 22:48:03.884499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:03.884499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 260/520] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:03 2022] GET /identity/v3/users/neutron => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:03.884499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8474f-43e6-449b-ba83-730fb9c52a3c None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:03.887852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3556a57-202e-45a2-a3e5-3c8759462616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:03.888533 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3556a57-202e-45a2-a3e5-3c8759462616 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:03.888823 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3556a57-202e-45a2-a3e5-3c8759462616 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:03.889103 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3556a57-202e-45a2-a3e5-3c8759462616 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:03.889809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3556a57-202e-45a2-a3e5-3c8759462616 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:03.891285 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3556a57-202e-45a2-a3e5-3c8759462616 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:03.918681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 261/521] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 22:48:03 2022] POST /identity/v3/users => generated 247 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:48:03.918681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3556a57-202e-45a2-a3e5-3c8759462616 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:04.016534 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab8474f-43e6-449b-ba83-730fb9c52a3c None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:04.030365 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3556a57-202e-45a2-a3e5-3c8759462616 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:04.048646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:04.057964 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2442ed-51bc-4bdb-99fc-bba84d3ff03d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:04.058659 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2442ed-51bc-4bdb-99fc-bba84d3ff03d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:04.058950 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2442ed-51bc-4bdb-99fc-bba84d3ff03d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:04.059234 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2442ed-51bc-4bdb-99fc-bba84d3ff03d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:04.073217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942354a-4618-4b42-9712-e98a4c300f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:04.073217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942354a-4618-4b42-9712-e98a4c300f6f None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:04.073217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942354a-4618-4b42-9712-e98a4c300f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:04.073217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942354a-4618-4b42-9712-e98a4c300f6f None admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:04.073622 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942354a-4618-4b42-9712-e98a4c300f6f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:04.073840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942354a-4618-4b42-9712-e98a4c300f6f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:04.074298 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:04.075384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0942354a-4618-4b42-9712-e98a4c300f6f None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:04.095805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 262/522] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:04.097296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942354a-4618-4b42-9712-e98a4c300f6f None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:04.151895 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2442ed-51bc-4bdb-99fc-bba84d3ff03d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:04.153681 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2442ed-51bc-4bdb-99fc-bba84d3ff03d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:04.212210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942354a-4618-4b42-9712-e98a4c300f6f None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:04.251230 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd464b6-51a9-474b-b1ba-fe9aa4b27c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:04.252026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd464b6-51a9-474b-b1ba-fe9aa4b27c50 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:04.252308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd464b6-51a9-474b-b1ba-fe9aa4b27c50 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:04.252588 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd464b6-51a9-474b-b1ba-fe9aa4b27c50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:04.259425 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 261/523] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:48:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:48:04.260415 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2442ed-51bc-4bdb-99fc-bba84d3ff03d None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:04.267698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd464b6-51a9-474b-b1ba-fe9aa4b27c50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:04.269555 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd464b6-51a9-474b-b1ba-fe9aa4b27c50 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:04.293451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-bcd464b6-51a9-474b-b1ba-fe9aa4b27c50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:04.293451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:04.293451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:04.293451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:04.293451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:04.293451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:04.293451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:04.293451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:04.293451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:04.293451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:04.293451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:04.293451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:04.293451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:04.293451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:04.293451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:04.294474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:04.294474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:04.294474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:04.294474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:04.294474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:04.294474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:04.294474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:04.294474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:04.294474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:04.294474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:04.294474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:04.294474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:04.294474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:04.294474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:04.294474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:04.294474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:04.294474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:04.295395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:04.295395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:04.295395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:04.295395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:04.295395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 263/524] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:04.295862 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd464b6-51a9-474b-b1ba-fe9aa4b27c50 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:04.308710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:04.329681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:04.376667 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2442ed-51bc-4bdb-99fc-bba84d3ff03d None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:04.408096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd464b6-51a9-474b-b1ba-fe9aa4b27c50 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:04.411371 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17351214-a422-46bc-8ef6-5d4990555884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:04.412299 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17351214-a422-46bc-8ef6-5d4990555884 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:04.412609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17351214-a422-46bc-8ef6-5d4990555884 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:04.412951 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17351214-a422-46bc-8ef6-5d4990555884 None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:04.413850 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17351214-a422-46bc-8ef6-5d4990555884 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:04.414222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17351214-a422-46bc-8ef6-5d4990555884 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:04.415753 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17351214-a422-46bc-8ef6-5d4990555884 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:04.435521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 262/525] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jun 16 22:48:04 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:04.436349 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17351214-a422-46bc-8ef6-5d4990555884 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:04.451215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab2b02-ee36-45c9-9f0d-0efb94a817a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:04.451873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab2b02-ee36-45c9-9f0d-0efb94a817a9 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:04.452156 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab2b02-ee36-45c9-9f0d-0efb94a817a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:04.452436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ab2b02-ee36-45c9-9f0d-0efb94a817a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:04.453303 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab2b02-ee36-45c9-9f0d-0efb94a817a9 None admin] RBAC: Adding query filter params (role.id=ec841a43f9924fa1b2209bc5ecea5e04, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=4180f2593149437fa27effd8d1338d10) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:04.453601 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab2b02-ee36-45c9-9f0d-0efb94a817a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:04.455084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ab2b02-ee36-45c9-9f0d-0efb94a817a9 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:04.462632 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 264/526] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:48:04 2022] GET /identity/v3/role_assignments?user.id=4180f2593149437fa27effd8d1338d10&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=ec841a43f9924fa1b2209bc5ecea5e04 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:04.462632 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab2b02-ee36-45c9-9f0d-0efb94a817a9 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:04.550797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:04.554941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17351214-a422-46bc-8ef6-5d4990555884 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:04.592360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68551274-441a-41d0-b521-eb675dedbedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:04.593397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68551274-441a-41d0-b521-eb675dedbedd None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:04.593847 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68551274-441a-41d0-b521-eb675dedbedd None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:04.594262 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68551274-441a-41d0-b521-eb675dedbedd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:04.602979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:04.606480 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68551274-441a-41d0-b521-eb675dedbedd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:04.607534 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ab2b02-ee36-45c9-9f0d-0efb94a817a9 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:04.608006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68551274-441a-41d0-b521-eb675dedbedd None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:04.627615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-68551274-441a-41d0-b521-eb675dedbedd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:04.627615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:04.627615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:04.627615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:04.627615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:04.627615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:04.627615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:04.627615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:04.627615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:04.627615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:04.627615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:04.627615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:04.627615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:04.627615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:04.627615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:04.628572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:04.628572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:04.628572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:04.628572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:04.628572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:04.628572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:04.628572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:04.628572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:04.628572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:04.628572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:04.628572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:04.628572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:04.628572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:04.628572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:04.628572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:04.628572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:04.628572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:04.629391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:04.629391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:04.629391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:04.629391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:04.629391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 263/527] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:04.630563 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68551274-441a-41d0-b521-eb675dedbedd None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:04.649981 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265afb1-322e-4290-a319-33977580e412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:04.651024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265afb1-322e-4290-a319-33977580e412 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:04.651522 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265afb1-322e-4290-a319-33977580e412 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:04.652001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7265afb1-322e-4290-a319-33977580e412 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:04.652822 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265afb1-322e-4290-a319-33977580e412 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:04.653289 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265afb1-322e-4290-a319-33977580e412 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:04.655050 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7265afb1-322e-4290-a319-33977580e412 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:04.668146 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 265/528] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:04.673387 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265afb1-322e-4290-a319-33977580e412 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:04.769671 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68551274-441a-41d0-b521-eb675dedbedd None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:04.779587 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7265afb1-322e-4290-a319-33977580e412 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:04.815044 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2533d9e-668c-472c-88a4-d120d5382e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:04.815505 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27cf5f-405a-41fb-af31-3a04a560c2c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:04.815957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2533d9e-668c-472c-88a4-d120d5382e17 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:04.816335 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2533d9e-668c-472c-88a4-d120d5382e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:04.816629 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27cf5f-405a-41fb-af31-3a04a560c2c1 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:04.816749 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2533d9e-668c-472c-88a4-d120d5382e17 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:04.817165 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27cf5f-405a-41fb-af31-3a04a560c2c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:04.817582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27cf5f-405a-41fb-af31-3a04a560c2c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:04.817801 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2533d9e-668c-472c-88a4-d120d5382e17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:04.818240 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2533d9e-668c-472c-88a4-d120d5382e17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:04.820240 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2533d9e-668c-472c-88a4-d120d5382e17 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:04.832252 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 266/529] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:04.833172 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2533d9e-668c-472c-88a4-d120d5382e17 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:04.844472 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:04.851553 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:04.911322 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27cf5f-405a-41fb-af31-3a04a560c2c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:04.913038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27cf5f-405a-41fb-af31-3a04a560c2c1 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:04.944099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2533d9e-668c-472c-88a4-d120d5382e17 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:04.987186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912177b-3322-4c2f-9577-23dcfba317ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:04.988141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912177b-3322-4c2f-9577-23dcfba317ef None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:04.988533 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912177b-3322-4c2f-9577-23dcfba317ef None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:04.988905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4912177b-3322-4c2f-9577-23dcfba317ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:04.996650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912177b-3322-4c2f-9577-23dcfba317ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:04.998491 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4912177b-3322-4c2f-9577-23dcfba317ef None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:05.014270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-4912177b-3322-4c2f-9577-23dcfba317ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:05.014270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:05.014270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:05.014270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:05.014270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:05.014270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:05.014270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:05.014270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:05.014270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:05.014270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:05.014270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:05.014270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:05.014270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:05.014270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:05.014270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:05.015301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:05.015301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:05.015301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:05.015301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:05.015301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:05.015301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:05.015301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:05.015301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:05.015301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:05.015301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:05.015301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:05.015301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:05.015301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:05.015301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:05.015301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:05.015301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:05.015301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:05.016330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:05.016330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:05.016330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:05.016330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:05.016330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:05.016330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:05.016330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:05.016330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:05.022660 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 267/530] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:05.023618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912177b-3322-4c2f-9577-23dcfba317ef None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:05.045098 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 264/531] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:48:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:48:05.047014 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27cf5f-405a-41fb-af31-3a04a560c2c1 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:05.112121 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:05.114192 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:05.149403 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4912177b-3322-4c2f-9577-23dcfba317ef None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:05.164129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27cf5f-405a-41fb-af31-3a04a560c2c1 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:05.188286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e93ce-f4e2-4235-acbe-a95d435bd3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:05.189443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e93ce-f4e2-4235-acbe-a95d435bd3df None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:05.189966 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e93ce-f4e2-4235-acbe-a95d435bd3df None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:05.190382 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5e93ce-f4e2-4235-acbe-a95d435bd3df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:05.198957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e93ce-f4e2-4235-acbe-a95d435bd3df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:05.200630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5e93ce-f4e2-4235-acbe-a95d435bd3df None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:05.203927 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a8db1-1d8d-45fc-8f5b-401d0c6081b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:05.204730 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a8db1-1d8d-45fc-8f5b-401d0c6081b8 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:05.205122 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a8db1-1d8d-45fc-8f5b-401d0c6081b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:05.205508 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3a8db1-1d8d-45fc-8f5b-401d0c6081b8 None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:05.206408 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a8db1-1d8d-45fc-8f5b-401d0c6081b8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:05.206856 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a8db1-1d8d-45fc-8f5b-401d0c6081b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:05.208410 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3a8db1-1d8d-45fc-8f5b-401d0c6081b8 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:05.216887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-0b5e93ce-f4e2-4235-acbe-a95d435bd3df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:05.216887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:05.216887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:05.216887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:05.216887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:05.216887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:05.216887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:05.216887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:05.216887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:05.216887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:05.216887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:05.216887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:05.216887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:05.216887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:05.216887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:05.217737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:05.217737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:05.217737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:05.217737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:05.217737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:05.217737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:05.217737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:05.217737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:05.217737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:05.217737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:05.217737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:05.217737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:05.217737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:05.217737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:05.217737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:05.217737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:05.217737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:05.218559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:05.218559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:05.218559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:05.218559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:05.218559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:05.218559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:05.218559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:05.218559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:05.220357 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 268/532] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:05.221328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e93ce-f4e2-4235-acbe-a95d435bd3df None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:05.233169 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 265/533] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:05.234047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a8db1-1d8d-45fc-8f5b-401d0c6081b8 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:05.326373 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5e93ce-f4e2-4235-acbe-a95d435bd3df None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:05.339838 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3a8db1-1d8d-45fc-8f5b-401d0c6081b8 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:05.357041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:05.363006 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cc91f-cc97-4450-ba32-dddb5848f350 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:05.364053 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cc91f-cc97-4450-ba32-dddb5848f350 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:05.364559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cc91f-cc97-4450-ba32-dddb5848f350 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:05.365060 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57cc91f-cc97-4450-ba32-dddb5848f350 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:05.376039 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cc91f-cc97-4450-ba32-dddb5848f350 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:05.377679 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad3d17-b244-45d5-a8cc-709939f59529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:05.377972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57cc91f-cc97-4450-ba32-dddb5848f350 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:05.378353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad3d17-b244-45d5-a8cc-709939f59529 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:05.378688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad3d17-b244-45d5-a8cc-709939f59529 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:05.379281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad3d17-b244-45d5-a8cc-709939f59529 None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:05.380179 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad3d17-b244-45d5-a8cc-709939f59529 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:05.380691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad3d17-b244-45d5-a8cc-709939f59529 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:05.386782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:05.387615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad3d17-b244-45d5-a8cc-709939f59529 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:05.397030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-d57cc91f-cc97-4450-ba32-dddb5848f350 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:05.397030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:05.397030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:05.397030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:05.397030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:05.397030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:05.397030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:05.397030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:05.397030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:05.397030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:05.397030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:05.397030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:05.397030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:05.397030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:05.397030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:05.397889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:05.397889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:05.397889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:05.397889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:05.397889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:05.397889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:05.397889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:05.397889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:05.397889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:05.397889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:05.397889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:05.397889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:05.397889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:05.397889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:05.397889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:05.397889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:05.397889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:05.398766 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:05.398766 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:05.398766 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:05.398766 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:05.402845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 269/535] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:05.403069 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 266/535] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:05.403891 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cc91f-cc97-4450-ba32-dddb5848f350 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:05.404659 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad3d17-b244-45d5-a8cc-709939f59529 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:05.507246 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57cc91f-cc97-4450-ba32-dddb5848f350 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:05.514644 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad3d17-b244-45d5-a8cc-709939f59529 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:05.542003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a949a-320e-45ad-8709-246dee50610a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:05.542812 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a949a-320e-45ad-8709-246dee50610a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:05.543020 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a949a-320e-45ad-8709-246dee50610a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:05.543285 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159a949a-320e-45ad-8709-246dee50610a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:05.544322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a949a-320e-45ad-8709-246dee50610a None admin] RBAC: Adding query filter params (role.id=ec841a43f9924fa1b2209bc5ecea5e04, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=915f4296504b47d796e3a5b99550eb75) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:05.544431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a949a-320e-45ad-8709-246dee50610a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:05.545848 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159a949a-320e-45ad-8709-246dee50610a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:05.558627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 270/536] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:48:05 2022] GET /identity/v3/role_assignments?user.id=915f4296504b47d796e3a5b99550eb75&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=ec841a43f9924fa1b2209bc5ecea5e04 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:05.558627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a949a-320e-45ad-8709-246dee50610a None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:05.561580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595ccdf-f53b-4c02-ae62-d237727f0aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:05.562104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595ccdf-f53b-4c02-ae62-d237727f0aad None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:05.562397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595ccdf-f53b-4c02-ae62-d237727f0aad None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:05.568281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1595ccdf-f53b-4c02-ae62-d237727f0aad None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:05.569263 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595ccdf-f53b-4c02-ae62-d237727f0aad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:05.569582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595ccdf-f53b-4c02-ae62-d237727f0aad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:05.571063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1595ccdf-f53b-4c02-ae62-d237727f0aad None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:05.594213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 267/537] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:05.603472 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595ccdf-f53b-4c02-ae62-d237727f0aad None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:05.660965 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:05.678981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:05.774050 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159a949a-320e-45ad-8709-246dee50610a None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:05.793264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1595ccdf-f53b-4c02-ae62-d237727f0aad None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:05.810950 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419a9ba-8a44-4b68-8aac-0ec45d78e21d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:05.811620 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419a9ba-8a44-4b68-8aac-0ec45d78e21d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:05.811911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419a9ba-8a44-4b68-8aac-0ec45d78e21d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:05.812195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f419a9ba-8a44-4b68-8aac-0ec45d78e21d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:05.813034 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419a9ba-8a44-4b68-8aac-0ec45d78e21d None admin] RBAC: Adding query filter params (role.id=ec841a43f9924fa1b2209bc5ecea5e04, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=24ca7f221334442781f10574277bf422) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:05.813362 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419a9ba-8a44-4b68-8aac-0ec45d78e21d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:05.814823 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f419a9ba-8a44-4b68-8aac-0ec45d78e21d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:05.821006 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 271/538] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:48:05 2022] GET /identity/v3/role_assignments?user.id=24ca7f221334442781f10574277bf422&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=ec841a43f9924fa1b2209bc5ecea5e04 => generated 259 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:05.872094 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf15be9-a6a5-442e-8c41-f724c91a5560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:05.872922 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf15be9-a6a5-442e-8c41-f724c91a5560 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:05.873208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf15be9-a6a5-442e-8c41-f724c91a5560 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:05.873487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf15be9-a6a5-442e-8c41-f724c91a5560 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:05.887478 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf15be9-a6a5-442e-8c41-f724c91a5560 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:05.888946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf15be9-a6a5-442e-8c41-f724c91a5560 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:05.906737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-ddf15be9-a6a5-442e-8c41-f724c91a5560 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:05.906737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:05.906737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:05.906737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:05.906737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:05.906737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:05.906737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:05.906737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:05.906737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:05.906737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:05.906737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:05.906737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:05.906737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:05.906737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:05.906737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:05.907645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:05.907645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:05.907645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:05.907645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:05.907645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:05.907645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:05.907645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:05.907645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:05.907645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:05.907645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:05.907645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:05.907645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:05.907645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:05.907645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:05.907645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:05.907645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:05.907645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:05.908466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:05.908466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:05.908466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:05.908466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:05.908466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:05.908466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:05.908466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:05.908466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:05.908466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 268/539] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:05.913760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419a9ba-8a44-4b68-8aac-0ec45d78e21d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:06.069514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f419a9ba-8a44-4b68-8aac-0ec45d78e21d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:06.083501 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:06.108815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211ccb46-d91d-4045-9623-f5902caf9c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:06.109495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ccb46-d91d-4045-9623-f5902caf9c9a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:06.109779 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ccb46-d91d-4045-9623-f5902caf9c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:06.110058 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211ccb46-d91d-4045-9623-f5902caf9c9a None admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:06.110890 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ccb46-d91d-4045-9623-f5902caf9c9a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:06.111205 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ccb46-d91d-4045-9623-f5902caf9c9a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:06.112682 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211ccb46-d91d-4045-9623-f5902caf9c9a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:06.126828 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 272/540] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:06.136966 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf15be9-a6a5-442e-8c41-f724c91a5560 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:06.183793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878345b-b55d-4240-a8fc-7b2d33bf9746 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:06.184104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878345b-b55d-4240-a8fc-7b2d33bf9746 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:06.184267 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878345b-b55d-4240-a8fc-7b2d33bf9746 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:06.185623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 273/541] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:48:06.198373 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4326d8-3233-435e-985e-d6c65326eb01 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:06.198686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4326d8-3233-435e-985e-d6c65326eb01 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:06.198969 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4326d8-3233-435e-985e-d6c65326eb01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:06.221423 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-5c4326d8-3233-435e-985e-d6c65326eb01 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:06.222643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5c4326d8-3233-435e-985e-d6c65326eb01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:06.295088 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:06.304728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf15be9-a6a5-442e-8c41-f724c91a5560 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:06.348011 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5712d-1a85-46d0-8404-071601587bff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:06.348674 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5712d-1a85-46d0-8404-071601587bff None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:06.350695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5712d-1a85-46d0-8404-071601587bff None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:06.350983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab5712d-1a85-46d0-8404-071601587bff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:06.351825 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5712d-1a85-46d0-8404-071601587bff None admin] RBAC: Adding query filter params (role.id=ec841a43f9924fa1b2209bc5ecea5e04, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=6b794f7aa2a7488e91f7d9ea35a45d78) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:06.352139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5712d-1a85-46d0-8404-071601587bff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:06.353599 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ab5712d-1a85-46d0-8404-071601587bff None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:06.366729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 274/542] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:48:06.369902 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 269/543] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:48:06 2022] GET /identity/v3/role_assignments?user.id=6b794f7aa2a7488e91f7d9ea35a45d78&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=ec841a43f9924fa1b2209bc5ecea5e04 => generated 259 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:06.387885 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4326d8-3233-435e-985e-d6c65326eb01 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:06.505972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4326d8-3233-435e-985e-d6c65326eb01 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:06.507922 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:06.545355 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758f8ab-591f-4a2b-bc41-796e2cb11bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:06.546233 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758f8ab-591f-4a2b-bc41-796e2cb11bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:06.546520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758f8ab-591f-4a2b-bc41-796e2cb11bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:06.546836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e758f8ab-591f-4a2b-bc41-796e2cb11bf0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:06.559071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e758f8ab-591f-4a2b-bc41-796e2cb11bf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:06.568444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-e758f8ab-591f-4a2b-bc41-796e2cb11bf0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:06.568444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:06.568444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:06.568444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:06.568444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:06.568444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:06.568444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:06.568444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:06.568444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:06.568444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:06.568444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:06.568444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:06.568444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:06.568444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:06.568444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:06.568444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:06.569384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:06.569384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:06.569384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:06.569384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:06.569384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:06.569384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:06.569384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:06.569384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:06.569384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:06.569384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:06.569384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:06.569384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:06.569384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:06.569384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:06.569384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:06.569384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:06.570167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:06.570167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:06.570167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:06.570167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:06.570167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:06.570167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:06.570167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:06.570167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:06.570167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:06.570167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:06.570167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:06.570167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:06.570167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 275/544] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:06.577208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5712d-1a85-46d0-8404-071601587bff None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:06.714421 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab5712d-1a85-46d0-8404-071601587bff None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:06.750990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a14d28-6f76-4138-8fdf-3b4300416c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:06.751663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a14d28-6f76-4138-8fdf-3b4300416c5d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:06.752007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a14d28-6f76-4138-8fdf-3b4300416c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:06.752224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a14d28-6f76-4138-8fdf-3b4300416c5d None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:06.752959 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a14d28-6f76-4138-8fdf-3b4300416c5d None admin] RBAC: Adding query filter params (name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:06.753267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a14d28-6f76-4138-8fdf-3b4300416c5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:06.754698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a14d28-6f76-4138-8fdf-3b4300416c5d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:06.763418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 270/545] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:48:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:06.771776 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758f8ab-591f-4a2b-bc41-796e2cb11bf0 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:06.894460 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e758f8ab-591f-4a2b-bc41-796e2cb11bf0 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:06.933737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024ee35-1b75-4136-884e-148b7b286483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:06.934548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024ee35-1b75-4136-884e-148b7b286483 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:06.934859 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024ee35-1b75-4136-884e-148b7b286483 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:06.935132 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024ee35-1b75-4136-884e-148b7b286483 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:06.946357 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024ee35-1b75-4136-884e-148b7b286483 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:06.947894 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024ee35-1b75-4136-884e-148b7b286483 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:06.965701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-3024ee35-1b75-4136-884e-148b7b286483 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:06.965701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:06.965701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:06.965701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:06.965701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:06.965701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:06.965701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:06.965701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:06.965701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:06.965701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:06.965701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:06.965701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:06.965701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:06.965701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:06.965701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:06.966582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:06.966582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:06.966582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:06.966582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:06.966582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:06.966582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:06.966582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:06.966582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:06.966582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:06.966582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:06.966582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:06.966582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:06.966582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:06.966582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:06.966582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:06.966582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:06.966582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:06.967457 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:06.967457 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:06.967457 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:06.967457 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:06.967457 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 276/546] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:06.976269 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a14d28-6f76-4138-8fdf-3b4300416c5d None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:06.978344 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bc0e9-824d-433e-b5de-0dd4ef09f263 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:06.978664 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bc0e9-824d-433e-b5de-0dd4ef09f263 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:06.978947 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428bc0e9-824d-433e-b5de-0dd4ef09f263 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:06.980257 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 277/547] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:06.985390 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe896ca9-61dd-4f2b-8101-3b0934112e4f None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:06.985684 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe896ca9-61dd-4f2b-8101-3b0934112e4f None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:06.985967 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe896ca9-61dd-4f2b-8101-3b0934112e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:06.998614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:07.003101 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-fe896ca9-61dd-4f2b-8101-3b0934112e4f None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:07.004303 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-fe896ca9-61dd-4f2b-8101-3b0934112e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:07.015025 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:07.082309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 278/548] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:07.108645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe896ca9-61dd-4f2b-8101-3b0934112e4f None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:07.160733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a14d28-6f76-4138-8fdf-3b4300416c5d None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:07.250021 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe896ca9-61dd-4f2b-8101-3b0934112e4f None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:07.284150 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa709f0a-4769-4fa0-ab1d-5e70055ec560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:07.284527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa709f0a-4769-4fa0-ab1d-5e70055ec560 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:07.284813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa709f0a-4769-4fa0-ab1d-5e70055ec560 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:07.285094 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa709f0a-4769-4fa0-ab1d-5e70055ec560 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:07.285837 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa709f0a-4769-4fa0-ab1d-5e70055ec560 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:07.286185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa709f0a-4769-4fa0-ab1d-5e70055ec560 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:07.287579 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa709f0a-4769-4fa0-ab1d-5e70055ec560 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:07.290638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5369e00-2059-4d66-b081-3203a0055eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:07.291838 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5369e00-2059-4d66-b081-3203a0055eca None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:07.292328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5369e00-2059-4d66-b081-3203a0055eca None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:07.292815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5369e00-2059-4d66-b081-3203a0055eca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:07.305795 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5369e00-2059-4d66-b081-3203a0055eca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:07.314559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-a5369e00-2059-4d66-b081-3203a0055eca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:07.314559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:07.314559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:07.314559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:07.314559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:07.314559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:07.314559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:07.314559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:07.314559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:07.314559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:07.314559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:07.314559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:07.314559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:07.314559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:07.314559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:07.314559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:07.315957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:07.315957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:07.315957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:07.315957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:07.315957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:07.315957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:07.315957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:07.315957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:07.315957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:07.315957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:07.315957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:07.315957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:07.315957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:07.315957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:07.315957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:07.315957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:07.316916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:07.316916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:07.316916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:07.316916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:07.316916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:07.316916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:07.316916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:07.316916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:07.316916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:07.316916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:07.316916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:07.316916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:07.319845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 279/549] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:07.325719 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 271/550] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:07.326999 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5369e00-2059-4d66-b081-3203a0055eca None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:07.331045 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:07.347651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa709f0a-4769-4fa0-ab1d-5e70055ec560 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:07.473090 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5369e00-2059-4d66-b081-3203a0055eca None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:07.503087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:07.550575 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791084b9-f491-4032-8427-9504bc32b5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:07.552252 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791084b9-f491-4032-8427-9504bc32b5fd None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:07.553877 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791084b9-f491-4032-8427-9504bc32b5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:07.554380 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791084b9-f491-4032-8427-9504bc32b5fd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:07.559395 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa709f0a-4769-4fa0-ab1d-5e70055ec560 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:07.568006 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791084b9-f491-4032-8427-9504bc32b5fd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:07.574457 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791084b9-f491-4032-8427-9504bc32b5fd None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:07.598570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-791084b9-f491-4032-8427-9504bc32b5fd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 22:48:07.598570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:07.598570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:07.598570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:07.598570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:07.598570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:07.598570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:07.598570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:07.598570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:07.598570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:07.598570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:07.598570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:07.598570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:07.598570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:07.598570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:07.598570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:07.599542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:07.599542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:07.599542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:07.599542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:07.599542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:07.599542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:07.599542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:07.599542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:07.599542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:07.599542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:07.599542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:07.599542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:07.599542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:07.599542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:07.599542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:07.599542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:07.599542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:07.599542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:07.601157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:07.601157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:07.601157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:07.601157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:07.601157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:07.601157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:07.601157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:07.601157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:07.601157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 22:48:07.601157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:07.609309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 280/551] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:48:07 2022] GET /identity/v3/users/swift => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:07.610171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791084b9-f491-4032-8427-9504bc32b5fd None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:07.636987 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49aadb-3fd8-49ce-9473-05773f27288b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:07.637662 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49aadb-3fd8-49ce-9473-05773f27288b None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:07.637949 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49aadb-3fd8-49ce-9473-05773f27288b None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:07.638230 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e49aadb-3fd8-49ce-9473-05773f27288b None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:07.639006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49aadb-3fd8-49ce-9473-05773f27288b None admin] RBAC: Adding query filter params (name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:07.639313 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49aadb-3fd8-49ce-9473-05773f27288b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:07.640697 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e49aadb-3fd8-49ce-9473-05773f27288b None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:07.656759 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 272/552] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:48:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:07.661416 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49aadb-3fd8-49ce-9473-05773f27288b None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:07.738932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:07.810090 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791084b9-f491-4032-8427-9504bc32b5fd None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:07.860960 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e49aadb-3fd8-49ce-9473-05773f27288b None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:07.886000 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b9087-de7b-410f-98e4-751ee828b4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:07.886718 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b9087-de7b-410f-98e4-751ee828b4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:07.887011 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b9087-de7b-410f-98e4-751ee828b4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:07.887296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90b9087-de7b-410f-98e4-751ee828b4b2 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:07.888155 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b9087-de7b-410f-98e4-751ee828b4b2 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:07.888474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b9087-de7b-410f-98e4-751ee828b4b2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:07.891414 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90b9087-de7b-410f-98e4-751ee828b4b2 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:07.935371 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bfc6f-2c0c-4693-adde-edee915ef531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:07.936192 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bfc6f-2c0c-4693-adde-edee915ef531 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:07.936478 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bfc6f-2c0c-4693-adde-edee915ef531 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:07.936754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70bfc6f-2c0c-4693-adde-edee915ef531 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:07.937581 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 281/553] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jun 16 22:48:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:07.938681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b9087-de7b-410f-98e4-751ee828b4b2 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:07.955444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:07.961334 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bfc6f-2c0c-4693-adde-edee915ef531 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:07.962838 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70bfc6f-2c0c-4693-adde-edee915ef531 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:07.982975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-b70bfc6f-2c0c-4693-adde-edee915ef531 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:07.982975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:07.982975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:07.982975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:07.982975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:07.982975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:07.982975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:07.982975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:07.982975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:07.982975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:07.982975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:07.982975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:07.982975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:07.982975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:07.982975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:07.983916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:07.983916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:07.983916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:07.983916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:07.983916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:07.983916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:07.983916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:07.983916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:07.983916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:07.983916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:07.983916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:07.983916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:07.983916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:07.983916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:07.983916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:07.983916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:07.983916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:07.984762 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:07.984762 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:07.984762 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:07.984762 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:07.985398 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 273/554] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:07.990547 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bfc6f-2c0c-4693-adde-edee915ef531 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:08.104972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70bfc6f-2c0c-4693-adde-edee915ef531 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:08.150673 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ea339-e64f-42a3-b6d2-1b91dc6392ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:08.151388 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ea339-e64f-42a3-b6d2-1b91dc6392ba None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:08.151676 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ea339-e64f-42a3-b6d2-1b91dc6392ba None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:08.151957 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ea339-e64f-42a3-b6d2-1b91dc6392ba None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:08.152679 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ea339-e64f-42a3-b6d2-1b91dc6392ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:08.152996 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ea339-e64f-42a3-b6d2-1b91dc6392ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:08.154635 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ea339-e64f-42a3-b6d2-1b91dc6392ba None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:08.172914 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 274/555] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:08.181072 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90b9087-de7b-410f-98e4-751ee828b4b2 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:08.182137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6e3f8-5790-45a0-bf44-307c4859aada None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:08.182731 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6e3f8-5790-45a0-bf44-307c4859aada None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:08.183238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b6e3f8-5790-45a0-bf44-307c4859aada None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:08.185179 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 275/556] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:08 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:08.189273 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6e3f8-5790-45a0-bf44-307c4859aada None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:08.212709 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:08.227732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8135d7e-6f5c-4930-b2cd-f91e7e751388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:08.228772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8135d7e-6f5c-4930-b2cd-f91e7e751388 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:08.229263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8135d7e-6f5c-4930-b2cd-f91e7e751388 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:08.229742 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8135d7e-6f5c-4930-b2cd-f91e7e751388 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:08.257016 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:08.306171 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b6e3f8-5790-45a0-bf44-307c4859aada None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:08.308346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8135d7e-6f5c-4930-b2cd-f91e7e751388 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:08.309821 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8135d7e-6f5c-4930-b2cd-f91e7e751388 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:08.342940 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc8da3c-5efe-48b5-acc0-548f137fdfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:08.343750 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8da3c-5efe-48b5-acc0-548f137fdfc1 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:08.344036 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8da3c-5efe-48b5-acc0-548f137fdfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:08.344317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc8da3c-5efe-48b5-acc0-548f137fdfc1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:08.352270 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8da3c-5efe-48b5-acc0-548f137fdfc1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:08.353709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc8da3c-5efe-48b5-acc0-548f137fdfc1 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:08.369251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-cfc8da3c-5efe-48b5-acc0-548f137fdfc1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 22:48:08.369251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:08.369251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:08.369251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:08.369251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:08.369251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:08.369251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:08.369251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:08.369251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:08.369251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:08.369251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:08.369251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:08.369251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:08.369251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:08.369251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:08.369251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:08.370638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:08.370638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:08.370638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:08.370638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:08.370638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:08.370638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:08.370638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:08.370638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:08.370638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:08.370638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:08.370638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:08.370638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:08.370638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:08.370638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:08.370638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:08.370638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:08.370638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:08.370638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:08.371571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:08.371571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:08.371571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:08.371571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:08.371571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:08.371571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:08.371571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:08.371571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:08.371571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 22:48:08.371571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:08.371571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 276/557] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jun 16 22:48:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:08.372179 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd95e8-dcd6-4a4a-94b3-9b2efe1ce591 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:08.372179 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd95e8-dcd6-4a4a-94b3-9b2efe1ce591 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:08.372179 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bd95e8-dcd6-4a4a-94b3-9b2efe1ce591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:08.388745 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-66bd95e8-dcd6-4a4a-94b3-9b2efe1ce591 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:08.389926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-66bd95e8-dcd6-4a4a-94b3-9b2efe1ce591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:08.425392 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 282/558] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:48:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:48:08.426879 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17909e06-cf1f-41c7-b04f-e4710f1eb99f None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:08.427244 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17909e06-cf1f-41c7-b04f-e4710f1eb99f None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:08.427615 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17909e06-cf1f-41c7-b04f-e4710f1eb99f None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:08.430629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 283/559] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:08.430629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17909e06-cf1f-41c7-b04f-e4710f1eb99f None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:08.455620 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:08.473013 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 277/560] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:08.474097 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd95e8-dcd6-4a4a-94b3-9b2efe1ce591 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:08.532995 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:08.549451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17909e06-cf1f-41c7-b04f-e4710f1eb99f None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:08.586232 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27df68f-e4d3-49d4-a2dd-f2d5fc2bdb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:08.587651 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27df68f-e4d3-49d4-a2dd-f2d5fc2bdb5c None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:08.588094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27df68f-e4d3-49d4-a2dd-f2d5fc2bdb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:08.588370 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27df68f-e4d3-49d4-a2dd-f2d5fc2bdb5c None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:08.589210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27df68f-e4d3-49d4-a2dd-f2d5fc2bdb5c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:08.589526 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27df68f-e4d3-49d4-a2dd-f2d5fc2bdb5c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:08.590983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27df68f-e4d3-49d4-a2dd-f2d5fc2bdb5c None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:08.591977 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bd95e8-dcd6-4a4a-94b3-9b2efe1ce591 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:08.608905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 284/561] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jun 16 22:48:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:08.610150 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76f4a1-3718-40f5-a814-20fbe1421ffa None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:08.610439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76f4a1-3718-40f5-a814-20fbe1421ffa None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:08.610753 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a76f4a1-3718-40f5-a814-20fbe1421ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:08.628027 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-5a76f4a1-3718-40f5-a814-20fbe1421ffa None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:08.629127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5a76f4a1-3718-40f5-a814-20fbe1421ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:08.634400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4306824-9e83-42e1-ab56-51656b43c329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:08.635475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4306824-9e83-42e1-ab56-51656b43c329 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:08.635910 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4306824-9e83-42e1-ab56-51656b43c329 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:08.636335 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4306824-9e83-42e1-ab56-51656b43c329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:08.648215 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4306824-9e83-42e1-ab56-51656b43c329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:08.650359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4306824-9e83-42e1-ab56-51656b43c329 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:08.668487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-d4306824-9e83-42e1-ab56-51656b43c329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:08.668487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:08.668487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:08.668487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:08.668487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:08.668487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:08.668487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:08.668487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:08.668487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:08.668487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:08.668487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:08.668487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:08.668487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:08.668487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:08.668487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:08.669362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:08.669362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:08.669362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:08.669362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:08.669362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:08.669362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:08.669362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:08.669362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:08.669362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:08.669362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:08.669362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:08.669362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:08.669362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:08.669362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:08.669362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:08.669362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:08.669362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:08.670224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:08.670224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:08.670224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:08.670224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:08.674347 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 278/562] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:08.674968 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4306824-9e83-42e1-ab56-51656b43c329 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:08.706635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 285/563] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:08.706635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76f4a1-3718-40f5-a814-20fbe1421ffa None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:08.722614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:08.790561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:08.799252 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4306824-9e83-42e1-ab56-51656b43c329 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:08.824041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a76f4a1-3718-40f5-a814-20fbe1421ffa None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:08.834516 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b704aa-262a-4d25-a678-6dc03cde32a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:08.835406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b704aa-262a-4d25-a678-6dc03cde32a1 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:08.835693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b704aa-262a-4d25-a678-6dc03cde32a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:08.835973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b704aa-262a-4d25-a678-6dc03cde32a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:08.846793 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b704aa-262a-4d25-a678-6dc03cde32a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:08.854515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-78b704aa-262a-4d25-a678-6dc03cde32a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:08.854515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:08.854515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:08.854515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:08.854515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:08.854515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:08.854515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:08.854515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:08.854515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:08.854515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:08.854515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:08.854515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:08.854515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:08.854515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:08.854515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:08.854515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:08.855587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:08.855587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:08.855587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:08.855587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:08.855587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:08.855587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:08.855587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:08.855587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:08.855587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:08.855587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:08.855587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:08.855587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:08.855587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:08.855587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:08.855587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:08.855587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:08.856582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:08.856582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:08.856582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:08.856582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:08.856582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:08.856582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:08.856582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:08.856582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:08.856582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:08.856582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:08.856582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:08.856582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:08.860518 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 279/564] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:08.861337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b704aa-262a-4d25-a678-6dc03cde32a1 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:08.869615 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80622e7f-2d70-4438-88c9-1ee0d872688a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:08.871178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80622e7f-2d70-4438-88c9-1ee0d872688a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:08.871618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80622e7f-2d70-4438-88c9-1ee0d872688a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:08.872041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80622e7f-2d70-4438-88c9-1ee0d872688a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:08.883584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80622e7f-2d70-4438-88c9-1ee0d872688a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:08.884948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80622e7f-2d70-4438-88c9-1ee0d872688a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:08.904284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-80622e7f-2d70-4438-88c9-1ee0d872688a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:08.904284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:08.904284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:08.904284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:08.904284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:08.904284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:08.904284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:08.904284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:08.904284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:08.904284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:08.904284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:08.904284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:08.904284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:08.904284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:08.904284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:08.905431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:08.905431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:08.905431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:08.905431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:08.905431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:08.905431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:08.905431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:08.905431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:08.905431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:08.905431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:08.905431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:08.905431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:08.905431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:08.905431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:08.905431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:08.905431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:08.905431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:08.906315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:08.906315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:08.906315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:08.906315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:08.906315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 286/565] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:08.906584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80622e7f-2d70-4438-88c9-1ee0d872688a None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:08.979354 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:08.979619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b704aa-262a-4d25-a678-6dc03cde32a1 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:09.014574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f650d-0c4d-4344-9620-ce8d8637d418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:09.015261 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f650d-0c4d-4344-9620-ce8d8637d418 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:09.015787 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f650d-0c4d-4344-9620-ce8d8637d418 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:09.015899 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4f650d-0c4d-4344-9620-ce8d8637d418 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:09.016533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f650d-0c4d-4344-9620-ce8d8637d418 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:09.016841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f650d-0c4d-4344-9620-ce8d8637d418 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:09.018241 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4f650d-0c4d-4344-9620-ce8d8637d418 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:09.018347 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80622e7f-2d70-4438-88c9-1ee0d872688a None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:09.030360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 280/566] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:09.031173 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f650d-0c4d-4344-9620-ce8d8637d418 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:09.046006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:09.057055 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd5513-c546-4dcb-9c2d-782a6d685932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:09.057853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd5513-c546-4dcb-9c2d-782a6d685932 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:09.058142 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd5513-c546-4dcb-9c2d-782a6d685932 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:09.058424 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd5513-c546-4dcb-9c2d-782a6d685932 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:09.068383 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dd5513-c546-4dcb-9c2d-782a6d685932 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:09.076075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-79dd5513-c546-4dcb-9c2d-782a6d685932 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:09.076075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:09.076075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:09.076075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:09.076075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:09.076075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:09.076075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:09.076075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:09.076075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:09.076075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:09.076075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:09.076075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:09.076075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:09.076075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:09.076075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:09.076075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:09.076986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:09.076986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:09.076986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:09.076986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:09.076986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:09.076986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:09.076986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:09.076986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:09.076986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:09.076986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:09.076986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:09.076986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:09.076986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:09.076986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:09.076986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:09.076986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:09.077796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:09.077796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:09.077796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:09.077796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:09.077796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:09.077796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:09.077796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:09.077796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:09.077796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:09.077796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:09.077796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:09.077796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:09.077796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 287/567] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:09.078488 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd5513-c546-4dcb-9c2d-782a6d685932 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:09.130067 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4f650d-0c4d-4344-9620-ce8d8637d418 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:09.164390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587bbcd0-1ac0-4701-80a9-2ac67723433d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:09.164996 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bbcd0-1ac0-4701-80a9-2ac67723433d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:09.165287 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bbcd0-1ac0-4701-80a9-2ac67723433d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:09.165574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587bbcd0-1ac0-4701-80a9-2ac67723433d None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:09.166766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bbcd0-1ac0-4701-80a9-2ac67723433d None admin] RBAC: Adding query filter params (name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:09.167239 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bbcd0-1ac0-4701-80a9-2ac67723433d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:09.168617 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587bbcd0-1ac0-4701-80a9-2ac67723433d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:09.176528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 281/568] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:48:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:09.177332 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dd5513-c546-4dcb-9c2d-782a6d685932 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:09.177858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc1119-6481-42d9-bdb7-05f8db2735f7 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:09.178154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc1119-6481-42d9-bdb7-05f8db2735f7 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:09.178437 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc1119-6481-42d9-bdb7-05f8db2735f7 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:09.179708 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 282/569] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:09.180393 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc1119-6481-42d9-bdb7-05f8db2735f7 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:09.202550 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:09.215975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065557b-68ed-4870-855e-bb69b6e5a313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:09.216641 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065557b-68ed-4870-855e-bb69b6e5a313 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:09.216926 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065557b-68ed-4870-855e-bb69b6e5a313 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:09.217210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7065557b-68ed-4870-855e-bb69b6e5a313 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:09.217929 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065557b-68ed-4870-855e-bb69b6e5a313 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:09.218237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065557b-68ed-4870-855e-bb69b6e5a313 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:09.219885 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7065557b-68ed-4870-855e-bb69b6e5a313 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:09.235987 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 288/570] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:09.238159 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065557b-68ed-4870-855e-bb69b6e5a313 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:09.266860 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:09.291382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc1119-6481-42d9-bdb7-05f8db2735f7 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:09.324050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadaeb4a-fd36-48fd-bde8-2f110fddd0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:09.324841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaeb4a-fd36-48fd-bde8-2f110fddd0eb None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:09.325126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaeb4a-fd36-48fd-bde8-2f110fddd0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:09.325402 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dadaeb4a-fd36-48fd-bde8-2f110fddd0eb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:09.331942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadaeb4a-fd36-48fd-bde8-2f110fddd0eb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:09.333386 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dadaeb4a-fd36-48fd-bde8-2f110fddd0eb None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:09.338579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7065557b-68ed-4870-855e-bb69b6e5a313 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:09.347339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-dadaeb4a-fd36-48fd-bde8-2f110fddd0eb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:09.347339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:09.347339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:09.347339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:09.347339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:09.347339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:09.347339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:09.347339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:09.347339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:09.347339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:09.347339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:09.347339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:09.347339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:09.347339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:09.347339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:09.348196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:09.348196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:09.348196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:09.348196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:09.348196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:09.348196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:09.348196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:09.348196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:09.348196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:09.348196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:09.348196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:09.348196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:09.348196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:09.348196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:09.348196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:09.348196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:09.348196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:09.349058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:09.349058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:09.349058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:09.349058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:09.349058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:09.349058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:09.349058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:09.349058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:09.349058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 283/571] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:09.350020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadaeb4a-fd36-48fd-bde8-2f110fddd0eb None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:09.374362 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9403bff1-4aa8-4ea5-8f63-2c8ff6489a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:09.375086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403bff1-4aa8-4ea5-8f63-2c8ff6489a5b None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:09.375372 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403bff1-4aa8-4ea5-8f63-2c8ff6489a5b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:09.375652 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9403bff1-4aa8-4ea5-8f63-2c8ff6489a5b None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:09.376393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9403bff1-4aa8-4ea5-8f63-2c8ff6489a5b None admin] RBAC: Adding query filter params (name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:09.376704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9403bff1-4aa8-4ea5-8f63-2c8ff6489a5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:09.378059 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9403bff1-4aa8-4ea5-8f63-2c8ff6489a5b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:09.386383 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 289/572] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:48:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:09.387700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0fa7c-8a31-4d64-9944-96aebb674985 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:09.387996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0fa7c-8a31-4d64-9944-96aebb674985 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:09.388282 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b0fa7c-8a31-4d64-9944-96aebb674985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:09.405973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-77b0fa7c-8a31-4d64-9944-96aebb674985 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:09.407175 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-77b0fa7c-8a31-4d64-9944-96aebb674985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:09.445900 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:09.448812 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dadaeb4a-fd36-48fd-bde8-2f110fddd0eb None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:09.473795 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 290/573] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:09.475232 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0fa7c-8a31-4d64-9944-96aebb674985 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:09.484094 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826617e-59ce-41b1-ae2b-1102c21a21b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:09.485251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826617e-59ce-41b1-ae2b-1102c21a21b9 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:09.485536 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826617e-59ce-41b1-ae2b-1102c21a21b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:09.486106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c826617e-59ce-41b1-ae2b-1102c21a21b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:09.495524 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:09.498023 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826617e-59ce-41b1-ae2b-1102c21a21b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:09.499898 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c826617e-59ce-41b1-ae2b-1102c21a21b9 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:09.517587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-c826617e-59ce-41b1-ae2b-1102c21a21b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:09.517587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:09.517587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:09.517587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:09.517587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:09.517587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:09.517587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:09.517587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:09.517587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:09.517587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:09.517587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:09.517587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:09.517587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:09.517587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:09.517587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:09.518527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:09.518527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:09.518527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:09.518527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:09.518527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:09.518527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:09.518527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:09.518527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:09.518527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:09.518527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:09.518527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:09.518527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:09.518527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:09.518527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:09.518527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:09.518527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:09.518527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:09.519457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:09.519457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:09.519457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:09.519457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:09.519457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 284/574] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:09.519729 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826617e-59ce-41b1-ae2b-1102c21a21b9 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:09.578182 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b0fa7c-8a31-4d64-9944-96aebb674985 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:09.612228 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8da195-3c81-49d6-b8cd-f11ede608e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:09.613017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8da195-3c81-49d6-b8cd-f11ede608e01 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:09.613304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8da195-3c81-49d6-b8cd-f11ede608e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:09.613581 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8da195-3c81-49d6-b8cd-f11ede608e01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:09.617442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c826617e-59ce-41b1-ae2b-1102c21a21b9 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:09.620135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8da195-3c81-49d6-b8cd-f11ede608e01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:09.621578 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8da195-3c81-49d6-b8cd-f11ede608e01 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:09.635049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-fe8da195-3c81-49d6-b8cd-f11ede608e01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:09.635049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:09.635049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:09.635049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:09.635049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:09.635049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:09.635049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:09.635049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:09.635049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:09.635049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:09.635049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:09.635049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:09.635049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:09.635049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:09.635049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:09.636107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:09.636107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:09.636107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:09.636107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:09.636107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:09.636107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:09.636107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:09.636107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:09.636107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:09.636107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:09.636107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:09.636107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:09.636107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:09.636107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:09.636107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:09.636107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:09.636107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:09.636972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:09.636972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:09.636972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:09.636972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:09.636972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:09.636972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:09.636972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:09.636972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:09.636972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 291/575] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:09.636972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8da195-3c81-49d6-b8cd-f11ede608e01 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:09.655097 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a0fe1-af9f-4e78-a7f1-d119d6ae1ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:09.655759 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a0fe1-af9f-4e78-a7f1-d119d6ae1ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:09.656049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a0fe1-af9f-4e78-a7f1-d119d6ae1ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:09.656329 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623a0fe1-af9f-4e78-a7f1-d119d6ae1ee6 None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:09.657102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a0fe1-af9f-4e78-a7f1-d119d6ae1ee6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:09.657411 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a0fe1-af9f-4e78-a7f1-d119d6ae1ee6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:09.658858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623a0fe1-af9f-4e78-a7f1-d119d6ae1ee6 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:09.670608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 285/576] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:09.671392 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a0fe1-af9f-4e78-a7f1-d119d6ae1ee6 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:09.672217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:09.721521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:09.740824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8da195-3c81-49d6-b8cd-f11ede608e01 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:09.772008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623a0fe1-af9f-4e78-a7f1-d119d6ae1ee6 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:09.773686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab802cc-60bb-4fa2-b687-31d2d2928c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:09.774469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab802cc-60bb-4fa2-b687-31d2d2928c7a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:09.774781 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab802cc-60bb-4fa2-b687-31d2d2928c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:09.775058 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab802cc-60bb-4fa2-b687-31d2d2928c7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:09.785423 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab802cc-60bb-4fa2-b687-31d2d2928c7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:09.786870 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab802cc-60bb-4fa2-b687-31d2d2928c7a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:09.804569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-8ab802cc-60bb-4fa2-b687-31d2d2928c7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:09.804569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:09.804569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:09.804569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:09.804569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:09.804569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:09.804569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:09.804569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:09.804569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:09.804569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:09.804569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:09.804569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:09.804569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:09.804569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:09.804569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:09.805638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:09.805638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:09.805638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:09.805638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:09.805638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:09.805638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:09.805638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:09.805638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:09.805638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:09.805638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:09.805638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:09.805638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:09.805638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:09.805638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:09.805638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:09.805638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:09.805638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:09.811304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:09.811304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:09.811304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:09.811304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:09.811573 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36b58e-901e-48cb-8397-71f8d913abfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:09.811573 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36b58e-901e-48cb-8397-71f8d913abfc None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:09.811573 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36b58e-901e-48cb-8397-71f8d913abfc None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:09.811818 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 292/577] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:09.811900 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c36b58e-901e-48cb-8397-71f8d913abfc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:09.811981 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab802cc-60bb-4fa2-b687-31d2d2928c7a None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:09.820988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c36b58e-901e-48cb-8397-71f8d913abfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:09.828549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-8c36b58e-901e-48cb-8397-71f8d913abfc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:09.828549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:09.828549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:09.828549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:09.828549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:09.828549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:09.828549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:09.828549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:09.828549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:09.828549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:09.828549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:09.828549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:09.828549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:09.828549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:09.828549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:09.828549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:09.829640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:09.829640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:09.829640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:09.829640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:09.829640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:09.829640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:09.829640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:09.829640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:09.829640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:09.829640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:09.829640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:09.829640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:09.829640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:09.829640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:09.829640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:09.829640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:09.830693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:09.830693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:09.830693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:09.830693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:09.830693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:09.830693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:09.830693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:09.830693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:09.830693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:09.830693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:09.830693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:09.830693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:09.830693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 286/578] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:09.831510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36b58e-901e-48cb-8397-71f8d913abfc None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:09.892765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:09.908165 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab802cc-60bb-4fa2-b687-31d2d2928c7a None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:09.929045 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c36b58e-901e-48cb-8397-71f8d913abfc None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:09.937629 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:09.941790 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2821e13-c40a-4c45-8062-129cab7261ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:09.942455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2821e13-c40a-4c45-8062-129cab7261ee None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:09.942779 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2821e13-c40a-4c45-8062-129cab7261ee None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:09.943064 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2821e13-c40a-4c45-8062-129cab7261ee None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:09.943780 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2821e13-c40a-4c45-8062-129cab7261ee None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:09.944088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2821e13-c40a-4c45-8062-129cab7261ee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:09.945455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2821e13-c40a-4c45-8062-129cab7261ee None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:09.957262 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 293/579] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:09.958050 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2821e13-c40a-4c45-8062-129cab7261ee None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:09.967756 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bce84-776a-49f6-9775-b4a0a7ff1cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:09.968453 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bce84-776a-49f6-9775-b4a0a7ff1cac None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:09.968744 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bce84-776a-49f6-9775-b4a0a7ff1cac None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:09.969028 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620bce84-776a-49f6-9775-b4a0a7ff1cac None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:09.969817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bce84-776a-49f6-9775-b4a0a7ff1cac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:09.970134 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bce84-776a-49f6-9775-b4a0a7ff1cac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:09.971638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620bce84-776a-49f6-9775-b4a0a7ff1cac None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:09.988582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 287/580] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:09.988582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bce84-776a-49f6-9775-b4a0a7ff1cac None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:10.061293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2821e13-c40a-4c45-8062-129cab7261ee None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:10.079589 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620bce84-776a-49f6-9775-b4a0a7ff1cac None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:10.095622 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e867c0-f5dc-4bd9-83be-437dbb3d4e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:10.096275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e867c0-f5dc-4bd9-83be-437dbb3d4e4b None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:10.096562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e867c0-f5dc-4bd9-83be-437dbb3d4e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:10.096839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e867c0-f5dc-4bd9-83be-437dbb3d4e4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:10.097684 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e867c0-f5dc-4bd9-83be-437dbb3d4e4b None admin] RBAC: Adding query filter params (role.id=ec841a43f9924fa1b2209bc5ecea5e04, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=bef518c9430e4274b1066241f821f4a1) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:10.097997 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e867c0-f5dc-4bd9-83be-437dbb3d4e4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:10.099447 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e867c0-f5dc-4bd9-83be-437dbb3d4e4b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:10.105318 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 294/581] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:48:09 2022] GET /identity/v3/role_assignments?user.id=bef518c9430e4274b1066241f821f4a1&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=ec841a43f9924fa1b2209bc5ecea5e04 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:10.106137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e867c0-f5dc-4bd9-83be-437dbb3d4e4b None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:10.115546 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6049feb-74e3-4a45-a835-433fea62825a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:10.115940 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6049feb-74e3-4a45-a835-433fea62825a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:10.116142 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6049feb-74e3-4a45-a835-433fea62825a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:10.116423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6049feb-74e3-4a45-a835-433fea62825a None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:10.117172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6049feb-74e3-4a45-a835-433fea62825a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:10.117488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6049feb-74e3-4a45-a835-433fea62825a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:10.118943 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6049feb-74e3-4a45-a835-433fea62825a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:10.120749 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:10.133330 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 288/582] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:10.134012 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6049feb-74e3-4a45-a835-433fea62825a None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:10.168636 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:10.229197 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e867c0-f5dc-4bd9-83be-437dbb3d4e4b None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:10.258276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6049feb-74e3-4a45-a835-433fea62825a None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:10.268454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410b4ce-c65d-44cb-835e-9790a664372a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:10.269208 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410b4ce-c65d-44cb-835e-9790a664372a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:10.269497 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410b4ce-c65d-44cb-835e-9790a664372a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:10.269772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8410b4ce-c65d-44cb-835e-9790a664372a None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:10.270519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410b4ce-c65d-44cb-835e-9790a664372a None admin] RBAC: Adding query filter params (name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:10.270862 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410b4ce-c65d-44cb-835e-9790a664372a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:10.272787 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8410b4ce-c65d-44cb-835e-9790a664372a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:10.281204 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 295/583] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:48:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:10.281986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410b4ce-c65d-44cb-835e-9790a664372a None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:10.304278 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a6905-14cd-42c9-bdff-7f5dedc0ba1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:10.305087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a6905-14cd-42c9-bdff-7f5dedc0ba1e None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:10.305715 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a6905-14cd-42c9-bdff-7f5dedc0ba1e None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:10.305824 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362a6905-14cd-42c9-bdff-7f5dedc0ba1e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:10.314263 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a6905-14cd-42c9-bdff-7f5dedc0ba1e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:10.315897 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362a6905-14cd-42c9-bdff-7f5dedc0ba1e None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:10.331220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-362a6905-14cd-42c9-bdff-7f5dedc0ba1e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 22:48:10.331220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:10.331220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:10.331220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:10.331220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:10.331220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:10.331220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:10.331220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:10.331220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:10.331220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:10.331220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:10.331220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:10.331220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:10.331220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:10.331220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:10.331220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:10.332152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:10.332152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:10.332152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:10.332152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:10.332152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:10.332152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:10.332152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:10.332152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:10.332152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:10.332152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:10.332152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:10.332152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:10.332152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:10.332152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:10.332152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:10.332152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:10.332152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:10.332152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:10.333080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:10.333080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:10.333080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:10.333080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:10.333080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:10.333080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:10.333080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:10.333080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:10.333080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 22:48:10.333080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:10.333080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 289/584] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:48:10 2022] GET /identity/v3/users/glance => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:10.333640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a6905-14cd-42c9-bdff-7f5dedc0ba1e None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:10.359225 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:10.390944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8410b4ce-c65d-44cb-835e-9790a664372a None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:10.424755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38166144-2abd-4b4d-9817-fe302d081978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:10.425650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38166144-2abd-4b4d-9817-fe302d081978 None admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:10.425939 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38166144-2abd-4b4d-9817-fe302d081978 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:10.426246 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38166144-2abd-4b4d-9817-fe302d081978 None admin] PATH_INFO: `/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/4180f2593149437fa27effd8d1338d10/roles/ec841a43f9924fa1b2209bc5ecea5e04` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:10.430154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:10.443328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362a6905-14cd-42c9-bdff-7f5dedc0ba1e None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:10.443818 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38166144-2abd-4b4d-9817-fe302d081978 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc48914270df4db996bd2a4b5b9fc777, user_id=4180f2593149437fa27effd8d1338d10, role_id=ec841a43f9924fa1b2209bc5ecea5e04)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:10.445266 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38166144-2abd-4b4d-9817-fe302d081978 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:10.458469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 296/585] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:48:10 2022] PUT /identity/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/4180f2593149437fa27effd8d1338d10/roles/ec841a43f9924fa1b2209bc5ecea5e04 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:48:10.459937 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38166144-2abd-4b4d-9817-fe302d081978 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:10.495146 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca16e1-2ec1-4e6c-b27b-1d949c02cf1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:10.496906 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca16e1-2ec1-4e6c-b27b-1d949c02cf1b None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:10.497432 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca16e1-2ec1-4e6c-b27b-1d949c02cf1b None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:10.498090 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bca16e1-2ec1-4e6c-b27b-1d949c02cf1b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:10.508064 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca16e1-2ec1-4e6c-b27b-1d949c02cf1b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:10.510042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bca16e1-2ec1-4e6c-b27b-1d949c02cf1b None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:10.527485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-6bca16e1-2ec1-4e6c-b27b-1d949c02cf1b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 22:48:10.527485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:10.527485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:10.527485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:10.527485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:10.527485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:10.527485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:10.527485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:10.527485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:10.527485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:10.527485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:10.527485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:10.527485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:10.527485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:10.527485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:10.527485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:10.528741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:10.528741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:10.528741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:10.528741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:10.528741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:10.528741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:10.528741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:10.528741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:10.528741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:10.528741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:10.528741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:10.528741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:10.528741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:10.528741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:10.528741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:10.528741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:10.528741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:10.528741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:10.529655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:10.529655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:10.529655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:10.529655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:10.529655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:10.529655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:10.529655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:10.529655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:10.529655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 22:48:10.529655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:10.532666 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 290/586] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:48:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:10.533903 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca16e1-2ec1-4e6c-b27b-1d949c02cf1b None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:10.582954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38166144-2abd-4b4d-9817-fe302d081978 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:10.612819 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:10.630265 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a5257-bcd6-4666-a9dd-ddd45f3cb67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:10.631657 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a5257-bcd6-4666-a9dd-ddd45f3cb67b None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:10.631939 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a5257-bcd6-4666-a9dd-ddd45f3cb67b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:10.632303 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303a5257-bcd6-4666-a9dd-ddd45f3cb67b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:10.643138 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a5257-bcd6-4666-a9dd-ddd45f3cb67b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:10.644629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303a5257-bcd6-4666-a9dd-ddd45f3cb67b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:10.655207 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bca16e1-2ec1-4e6c-b27b-1d949c02cf1b None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:10.664996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-303a5257-bcd6-4666-a9dd-ddd45f3cb67b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:10.664996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:10.664996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:10.664996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:10.664996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:10.664996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:10.664996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:10.664996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:10.664996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:10.664996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:10.664996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:10.664996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:10.664996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:10.664996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:10.664996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:10.665908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:10.665908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:10.665908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:10.665908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:10.665908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:10.665908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:10.665908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:10.665908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:10.665908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:10.665908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:10.665908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:10.665908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:10.665908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:10.665908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:10.665908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:10.665908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:10.665908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:10.667193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:10.667193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:10.667193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:10.667193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:10.667193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 297/587] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:10.667489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a5257-bcd6-4666-a9dd-ddd45f3cb67b None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:10.697598 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d2dbe-ee36-4909-a482-a6aca08a70b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:10.700105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d2dbe-ee36-4909-a482-a6aca08a70b0 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:10.700851 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d2dbe-ee36-4909-a482-a6aca08a70b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:10.701545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527d2dbe-ee36-4909-a482-a6aca08a70b0 None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:10.704110 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d2dbe-ee36-4909-a482-a6aca08a70b0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:10.704672 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d2dbe-ee36-4909-a482-a6aca08a70b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:10.706302 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527d2dbe-ee36-4909-a482-a6aca08a70b0 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:10.730809 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 291/588] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jun 16 22:48:10 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:10.731952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d2dbe-ee36-4909-a482-a6aca08a70b0 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:10.747993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:10.776533 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303a5257-bcd6-4666-a9dd-ddd45f3cb67b None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:10.814149 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457148a-85dd-4abf-8040-6d8062fc5014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:10.814976 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457148a-85dd-4abf-8040-6d8062fc5014 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:10.815268 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457148a-85dd-4abf-8040-6d8062fc5014 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:10.815545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a457148a-85dd-4abf-8040-6d8062fc5014 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:10.816508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457148a-85dd-4abf-8040-6d8062fc5014 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:10.816823 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457148a-85dd-4abf-8040-6d8062fc5014 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:10.818921 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a457148a-85dd-4abf-8040-6d8062fc5014 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:10.838676 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 298/589] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jun 16 22:48:10 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:10.840503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457148a-85dd-4abf-8040-6d8062fc5014 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:10.867395 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527d2dbe-ee36-4909-a482-a6aca08a70b0 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:10.875907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:10.919026 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4dd22-ab59-4a75-afaf-20d7e1618605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:10.920360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4dd22-ab59-4a75-afaf-20d7e1618605 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:10.921679 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4dd22-ab59-4a75-afaf-20d7e1618605 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:10.922166 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea4dd22-ab59-4a75-afaf-20d7e1618605 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:10.923135 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4dd22-ab59-4a75-afaf-20d7e1618605 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:10.923642 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4dd22-ab59-4a75-afaf-20d7e1618605 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:10.925214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea4dd22-ab59-4a75-afaf-20d7e1618605 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:10.942181 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 292/590] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:10.943984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4dd22-ab59-4a75-afaf-20d7e1618605 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:10.953154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a457148a-85dd-4abf-8040-6d8062fc5014 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:11.001191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b53fd3-fbc8-4872-9654-ff65a11d985b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:11.003000 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b53fd3-fbc8-4872-9654-ff65a11d985b None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:11.003843 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b53fd3-fbc8-4872-9654-ff65a11d985b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:11.004691 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b53fd3-fbc8-4872-9654-ff65a11d985b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:11.019728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b53fd3-fbc8-4872-9654-ff65a11d985b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:11.022750 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b53fd3-fbc8-4872-9654-ff65a11d985b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:11.028647 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:11.046096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-07b53fd3-fbc8-4872-9654-ff65a11d985b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:11.046096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:11.046096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:11.046096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:11.046096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:11.046096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:11.046096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:11.046096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:11.046096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:11.046096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:11.046096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:11.046096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:11.046096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:11.046096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:11.046096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:11.047041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:11.047041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:11.047041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:11.047041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:11.047041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:11.047041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:11.047041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:11.047041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:11.047041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:11.047041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:11.047041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:11.047041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:11.047041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:11.047041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:11.047041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:11.047041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:11.047041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:11.047935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:11.047935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:11.047935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:11.047935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:11.051966 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 299/591] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:11.053336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b53fd3-fbc8-4872-9654-ff65a11d985b None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:11.075192 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea4dd22-ab59-4a75-afaf-20d7e1618605 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:11.118869 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81a360-db41-4dfa-ae1d-0d7f95c4f440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:11.119805 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81a360-db41-4dfa-ae1d-0d7f95c4f440 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:11.120095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81a360-db41-4dfa-ae1d-0d7f95c4f440 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:11.120374 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea81a360-db41-4dfa-ae1d-0d7f95c4f440 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:11.131420 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81a360-db41-4dfa-ae1d-0d7f95c4f440 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:11.132872 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea81a360-db41-4dfa-ae1d-0d7f95c4f440 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:11.150209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:11.150654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-ea81a360-db41-4dfa-ae1d-0d7f95c4f440 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:11.150654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:11.150654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:11.150654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:11.150654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:11.150654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:11.150654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:11.150654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:11.150654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:11.150654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:11.150654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:11.150654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:11.150654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:11.150654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:11.150654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:11.151533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:11.151533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:11.151533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:11.151533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:11.151533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:11.151533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:11.151533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:11.151533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:11.151533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:11.151533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:11.151533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:11.151533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:11.151533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:11.151533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:11.151533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:11.151533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:11.151533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:11.152408 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:11.152408 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:11.152408 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:11.152408 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:11.156610 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 293/592] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:11.157576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81a360-db41-4dfa-ae1d-0d7f95c4f440 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:11.173029 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b53fd3-fbc8-4872-9654-ff65a11d985b None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:11.213849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31853a45-2668-41b6-879a-13261113e947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:11.215082 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31853a45-2668-41b6-879a-13261113e947 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:11.215584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31853a45-2668-41b6-879a-13261113e947 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:11.216073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31853a45-2668-41b6-879a-13261113e947 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:11.224738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31853a45-2668-41b6-879a-13261113e947 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:11.226582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31853a45-2668-41b6-879a-13261113e947 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:11.245069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-31853a45-2668-41b6-879a-13261113e947 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 22:48:11.245069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:11.245069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:11.245069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:11.245069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:11.245069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:11.245069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:11.245069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:11.245069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:11.245069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:11.245069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:11.245069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:11.245069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:11.245069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:11.245069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:11.245069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:11.245993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:11.245993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:11.245993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:11.245993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:11.245993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:11.245993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:11.245993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:11.245993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:11.245993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:11.245993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:11.245993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:11.245993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:11.245993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:11.245993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:11.245993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:11.245993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:11.245993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:11.245993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:11.246946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:11.246946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:11.246946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:11.246946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:11.246946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:11.246946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:11.246946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:11.246946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:11.246946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 22:48:11.246946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:11.249659 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 300/593] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:11 2022] GET /identity/v3/users/neutron => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:11.250770 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31853a45-2668-41b6-879a-13261113e947 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:11.263620 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea81a360-db41-4dfa-ae1d-0d7f95c4f440 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:11.281790 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:11.300655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153b8de-5138-4139-8d4e-30909fbb72bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:11.301268 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153b8de-5138-4139-8d4e-30909fbb72bd None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:11.301563 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153b8de-5138-4139-8d4e-30909fbb72bd None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:11.301843 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8153b8de-5138-4139-8d4e-30909fbb72bd None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:11.302675 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153b8de-5138-4139-8d4e-30909fbb72bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:11.302996 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153b8de-5138-4139-8d4e-30909fbb72bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:11.304559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8153b8de-5138-4139-8d4e-30909fbb72bd None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:11.317026 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 294/594] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:11.322938 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153b8de-5138-4139-8d4e-30909fbb72bd None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:11.366786 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31853a45-2668-41b6-879a-13261113e947 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:11.404657 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c0653-35e2-426d-9aab-7b23b076b7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:11.405753 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c0653-35e2-426d-9aab-7b23b076b7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:11.406284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c0653-35e2-426d-9aab-7b23b076b7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:11.406811 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6c0653-35e2-426d-9aab-7b23b076b7d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:11.407881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c0653-35e2-426d-9aab-7b23b076b7d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:11.408485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c0653-35e2-426d-9aab-7b23b076b7d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:11.410093 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6c0653-35e2-426d-9aab-7b23b076b7d7 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:11.423279 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 301/595] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:11.424338 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c0653-35e2-426d-9aab-7b23b076b7d7 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:11.428077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:11.428338 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8153b8de-5138-4139-8d4e-30909fbb72bd None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:11.464003 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aeb6be-6ec4-4e07-8463-db21c8033df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:11.464740 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aeb6be-6ec4-4e07-8463-db21c8033df3 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:11.464964 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aeb6be-6ec4-4e07-8463-db21c8033df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:11.465254 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1aeb6be-6ec4-4e07-8463-db21c8033df3 None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:11.466082 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aeb6be-6ec4-4e07-8463-db21c8033df3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:11.467071 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aeb6be-6ec4-4e07-8463-db21c8033df3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:11.467905 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1aeb6be-6ec4-4e07-8463-db21c8033df3 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:11.486099 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 295/596] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jun 16 22:48:11 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:11.486805 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aeb6be-6ec4-4e07-8463-db21c8033df3 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:11.521730 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:11.541951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6c0653-35e2-426d-9aab-7b23b076b7d7 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:11.582810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b49f5-adb6-4d04-9196-7125115ea153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:11.584304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b49f5-adb6-4d04-9196-7125115ea153 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:11.585176 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b49f5-adb6-4d04-9196-7125115ea153 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:11.585763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051b49f5-adb6-4d04-9196-7125115ea153 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:11.589754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1aeb6be-6ec4-4e07-8463-db21c8033df3 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:11.595458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b49f5-adb6-4d04-9196-7125115ea153 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:11.597249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051b49f5-adb6-4d04-9196-7125115ea153 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:11.613994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-051b49f5-adb6-4d04-9196-7125115ea153 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:11.613994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:11.613994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:11.613994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:11.613994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:11.613994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:11.613994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:11.613994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:11.613994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:11.613994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:11.613994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:11.613994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:11.613994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:11.613994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:11.613994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:11.616191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:11.616191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:11.616191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:11.616191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:11.616191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:11.616191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:11.616191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:11.616191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:11.616191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:11.616191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:11.616191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:11.616191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:11.616191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:11.616191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:11.616191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:11.616191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:11.616191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:11.617054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:11.617054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:11.617054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:11.617054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:11.617054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:11.617054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:11.617054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:11.617054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:11.619510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 302/597] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:11.620555 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b49f5-adb6-4d04-9196-7125115ea153 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:11.632628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb3f5e-a805-4883-8793-0ca1c24fbab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:11.633436 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb3f5e-a805-4883-8793-0ca1c24fbab2 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:11.633712 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb3f5e-a805-4883-8793-0ca1c24fbab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:11.633991 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb3f5e-a805-4883-8793-0ca1c24fbab2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:11.641129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb3f5e-a805-4883-8793-0ca1c24fbab2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:11.642578 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb3f5e-a805-4883-8793-0ca1c24fbab2 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:11.657133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-b8cb3f5e-a805-4883-8793-0ca1c24fbab2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:11.657133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:11.657133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:11.657133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:11.657133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:11.657133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:11.657133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:11.657133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:11.657133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:11.657133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:11.657133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:11.657133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:11.657133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:11.657133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:11.657133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:11.658542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:11.658542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:11.658542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:11.658542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:11.658542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:11.658542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:11.658542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:11.658542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:11.658542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:11.658542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:11.658542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:11.658542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:11.658542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:11.658542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:11.658542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:11.658542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:11.658542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:11.659423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:11.659423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:11.659423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:11.659423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:11.659423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:11.659423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:11.659423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:11.659423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:11.662995 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 296/598] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:11.663870 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb3f5e-a805-4883-8793-0ca1c24fbab2 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:11.697114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:11.739699 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051b49f5-adb6-4d04-9196-7125115ea153 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:11.762063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:11.767766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb3f5e-a805-4883-8793-0ca1c24fbab2 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:11.781449 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4550635-6f39-43c5-8591-552ba069241a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:11.782634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4550635-6f39-43c5-8591-552ba069241a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:11.783212 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4550635-6f39-43c5-8591-552ba069241a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:11.783694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4550635-6f39-43c5-8591-552ba069241a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:11.795303 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4550635-6f39-43c5-8591-552ba069241a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:11.797045 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4550635-6f39-43c5-8591-552ba069241a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:11.803834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb3d5b-9373-4d02-a9fc-f24b3604022e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:11.804506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb3d5b-9373-4d02-a9fc-f24b3604022e None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:11.804974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb3d5b-9373-4d02-a9fc-f24b3604022e None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:11.805263 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0eb3d5b-9373-4d02-a9fc-f24b3604022e None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:11.806055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb3d5b-9373-4d02-a9fc-f24b3604022e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:11.806372 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb3d5b-9373-4d02-a9fc-f24b3604022e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:11.807834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0eb3d5b-9373-4d02-a9fc-f24b3604022e None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:11.819255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-c4550635-6f39-43c5-8591-552ba069241a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:11.819255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:11.819255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:11.819255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:11.819255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:11.819255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:11.819255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:11.819255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:11.819255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:11.819255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:11.819255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:11.819255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:11.819255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:11.819255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:11.819255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:11.820105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:11.820105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:11.820105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:11.820105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:11.820105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:11.820105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:11.820105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:11.820105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:11.820105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:11.820105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:11.820105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:11.820105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:11.820105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:11.820105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:11.820105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:11.820105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:11.820105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:11.820995 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:11.820995 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:11.820995 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:11.820995 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:11.821865 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 297/599] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:11.823549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 303/600] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:11.830636 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb3d5b-9373-4d02-a9fc-f24b3604022e None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:11.830775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4550635-6f39-43c5-8591-552ba069241a None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:11.937859 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0eb3d5b-9373-4d02-a9fc-f24b3604022e None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:11.966552 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4550635-6f39-43c5-8591-552ba069241a None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:11.974691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af202bd3-1a4f-4f54-804f-552e1e9549ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:11.975368 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af202bd3-1a4f-4f54-804f-552e1e9549ed None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:11.975657 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af202bd3-1a4f-4f54-804f-552e1e9549ed None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:11.975941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af202bd3-1a4f-4f54-804f-552e1e9549ed None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:11.976720 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af202bd3-1a4f-4f54-804f-552e1e9549ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:11.977375 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af202bd3-1a4f-4f54-804f-552e1e9549ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:11.978466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af202bd3-1a4f-4f54-804f-552e1e9549ed None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:11.987484 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:11.996389 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 298/601] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:11.997293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af202bd3-1a4f-4f54-804f-552e1e9549ed None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:12.009292 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01bf6d-b7e5-436c-a746-6d3b255dd95c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:12.010311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01bf6d-b7e5-436c-a746-6d3b255dd95c None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:12.010778 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01bf6d-b7e5-436c-a746-6d3b255dd95c None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:12.011205 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e01bf6d-b7e5-436c-a746-6d3b255dd95c None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:12.012078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01bf6d-b7e5-436c-a746-6d3b255dd95c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:12.012573 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01bf6d-b7e5-436c-a746-6d3b255dd95c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:12.014251 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e01bf6d-b7e5-436c-a746-6d3b255dd95c None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:12.016245 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:12.033938 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 304/602] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:12.035211 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01bf6d-b7e5-436c-a746-6d3b255dd95c None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:12.103170 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af202bd3-1a4f-4f54-804f-552e1e9549ed None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:12.139029 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0f790-5d2b-4959-b726-0a9d96619250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:12.140007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0f790-5d2b-4959-b726-0a9d96619250 None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:12.140261 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0f790-5d2b-4959-b726-0a9d96619250 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:12.140535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf0f790-5d2b-4959-b726-0a9d96619250 None admin] PATH_INFO: `/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/915f4296504b47d796e3a5b99550eb75/roles/ec841a43f9924fa1b2209bc5ecea5e04` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:12.147887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e01bf6d-b7e5-436c-a746-6d3b255dd95c None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:12.155934 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0f790-5d2b-4959-b726-0a9d96619250 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc48914270df4db996bd2a4b5b9fc777, user_id=915f4296504b47d796e3a5b99550eb75, role_id=ec841a43f9924fa1b2209bc5ecea5e04)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:12.157457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf0f790-5d2b-4959-b726-0a9d96619250 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:12.178767 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 299/603] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:48:11 2022] PUT /identity/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/915f4296504b47d796e3a5b99550eb75/roles/ec841a43f9924fa1b2209bc5ecea5e04 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:48:12.180053 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0f790-5d2b-4959-b726-0a9d96619250 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:12.209541 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52596511-a3dc-4a5c-b3ba-b20d5c597c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:12.212508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52596511-a3dc-4a5c-b3ba-b20d5c597c60 None admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:12.212973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52596511-a3dc-4a5c-b3ba-b20d5c597c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:12.213496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52596511-a3dc-4a5c-b3ba-b20d5c597c60 None admin] PATH_INFO: `/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/24ca7f221334442781f10574277bf422/roles/ec841a43f9924fa1b2209bc5ecea5e04` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:12.253737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52596511-a3dc-4a5c-b3ba-b20d5c597c60 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc48914270df4db996bd2a4b5b9fc777, user_id=24ca7f221334442781f10574277bf422, role_id=ec841a43f9924fa1b2209bc5ecea5e04)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:12.257551 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52596511-a3dc-4a5c-b3ba-b20d5c597c60 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:12.290227 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 305/604] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:48:12 2022] PUT /identity/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/24ca7f221334442781f10574277bf422/roles/ec841a43f9924fa1b2209bc5ecea5e04 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:48:12.314531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2849045-76c2-4e1d-8f6e-740f6c1ee5ef None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:12.315004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2849045-76c2-4e1d-8f6e-740f6c1ee5ef None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:12.315383 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2849045-76c2-4e1d-8f6e-740f6c1ee5ef None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:12.317320 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 306/605] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:48:12.324048 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e625604-273a-45fb-8ecd-c26e29bbc719 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:12.324413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e625604-273a-45fb-8ecd-c26e29bbc719 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:12.324773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e625604-273a-45fb-8ecd-c26e29bbc719 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:12.332252 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:12.344324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-8e625604-273a-45fb-8ecd-c26e29bbc719 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:12.345578 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-8e625604-273a-45fb-8ecd-c26e29bbc719 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:12.369867 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:12.408516 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf0f790-5d2b-4959-b726-0a9d96619250 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:12.452910 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d335fdc-bbc4-4f8d-a9b3-bf07fadf7964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:12.454160 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d335fdc-bbc4-4f8d-a9b3-bf07fadf7964 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:12.454625 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d335fdc-bbc4-4f8d-a9b3-bf07fadf7964 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:12.455189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d335fdc-bbc4-4f8d-a9b3-bf07fadf7964 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:12.457281 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 307/606] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:48:12.464412 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d335fdc-bbc4-4f8d-a9b3-bf07fadf7964 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:12.465870 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d335fdc-bbc4-4f8d-a9b3-bf07fadf7964 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:12.480623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-0d335fdc-bbc4-4f8d-a9b3-bf07fadf7964 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:12.480623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:12.480623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:12.480623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:12.480623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:12.480623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:12.480623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:12.480623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:12.480623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:12.480623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:12.480623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:12.480623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:12.480623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:12.480623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:12.480623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:12.481558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:12.481558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:12.481558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:12.481558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:12.481558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:12.481558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:12.481558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:12.481558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:12.481558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:12.481558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:12.481558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:12.481558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:12.481558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:12.481558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:12.481558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:12.481558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:12.481558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:12.482919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:12.482919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:12.482919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:12.482919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:12.482919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:12.482919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:12.482919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:12.482919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:12.482919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 300/607] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:12.486780 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e625604-273a-45fb-8ecd-c26e29bbc719 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:12.487884 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d335fdc-bbc4-4f8d-a9b3-bf07fadf7964 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:12.627071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e625604-273a-45fb-8ecd-c26e29bbc719 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:12.627630 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d335fdc-bbc4-4f8d-a9b3-bf07fadf7964 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:12.685211 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:12.685819 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9991378d-a858-4f70-8e91-e0272224e78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:12.686411 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991378d-a858-4f70-8e91-e0272224e78c None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:12.686729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991378d-a858-4f70-8e91-e0272224e78c None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:12.687014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9991378d-a858-4f70-8e91-e0272224e78c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:12.696449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c998d-64fa-4b2e-94c5-5a21c99153fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:12.697403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c998d-64fa-4b2e-94c5-5a21c99153fe None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:12.697889 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c998d-64fa-4b2e-94c5-5a21c99153fe None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:12.698378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1c998d-64fa-4b2e-94c5-5a21c99153fe None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:12.699428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c998d-64fa-4b2e-94c5-5a21c99153fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:12.699966 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c998d-64fa-4b2e-94c5-5a21c99153fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:12.700611 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9991378d-a858-4f70-8e91-e0272224e78c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:12.701575 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1c998d-64fa-4b2e-94c5-5a21c99153fe None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:12.710436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-9991378d-a858-4f70-8e91-e0272224e78c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:12.710436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:12.710436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:12.710436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:12.710436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:12.710436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:12.710436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:12.710436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:12.710436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:12.710436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:12.710436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:12.710436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:12.710436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:12.710436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:12.710436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:12.710436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:12.711563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:12.711563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:12.711563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:12.711563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:12.711563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:12.711563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:12.711563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:12.711563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:12.711563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:12.711563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:12.711563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:12.711563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:12.711563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:12.711563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:12.711563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:12.711563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:12.712465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:12.712465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:12.712465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:12.712465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:12.712465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:12.712465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:12.712465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:12.712465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:12.712465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:12.712465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:12.712465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:12.712465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:12.715476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 308/608] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:12.721358 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 301/609] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:12.721496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35190fa4-9bb9-4b07-a01a-ad8acc4f7905 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:12.721496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35190fa4-9bb9-4b07-a01a-ad8acc4f7905 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:12.721496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35190fa4-9bb9-4b07-a01a-ad8acc4f7905 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:12.721496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 309/610] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:12.723554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c998d-64fa-4b2e-94c5-5a21c99153fe None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:12.725943 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35190fa4-9bb9-4b07-a01a-ad8acc4f7905 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:12.745618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:12.839318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1c998d-64fa-4b2e-94c5-5a21c99153fe None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:12.845326 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35190fa4-9bb9-4b07-a01a-ad8acc4f7905 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:12.877547 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f397ff-0783-4f54-8c3e-f48ca89dd8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:12.879521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f397ff-0783-4f54-8c3e-f48ca89dd8c4 None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:12.880008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f397ff-0783-4f54-8c3e-f48ca89dd8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:12.880484 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f397ff-0783-4f54-8c3e-f48ca89dd8c4 None admin] PATH_INFO: `/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/6b794f7aa2a7488e91f7d9ea35a45d78/roles/ec841a43f9924fa1b2209bc5ecea5e04` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:12.883345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65be1a91-99df-458b-afa8-18b3670fbfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:12.884090 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be1a91-99df-458b-afa8-18b3670fbfab None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:12.884306 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be1a91-99df-458b-afa8-18b3670fbfab None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:12.884591 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65be1a91-99df-458b-afa8-18b3670fbfab None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:12.885338 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be1a91-99df-458b-afa8-18b3670fbfab None admin] RBAC: Adding query filter params (name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:12.886379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be1a91-99df-458b-afa8-18b3670fbfab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:12.887078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65be1a91-99df-458b-afa8-18b3670fbfab None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:12.900099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 310/611] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:48:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:12.900099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453d74d-9d5b-4cf7-bae9-a9ce1c2abea5 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:12.900099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453d74d-9d5b-4cf7-bae9-a9ce1c2abea5 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:12.900099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453d74d-9d5b-4cf7-bae9-a9ce1c2abea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:12.908722 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f397ff-0783-4f54-8c3e-f48ca89dd8c4 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc48914270df4db996bd2a4b5b9fc777, user_id=6b794f7aa2a7488e91f7d9ea35a45d78, role_id=ec841a43f9924fa1b2209bc5ecea5e04)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:12.912414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f397ff-0783-4f54-8c3e-f48ca89dd8c4 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:12.923609 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-9453d74d-9d5b-4cf7-bae9-a9ce1c2abea5 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:12.924715 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-9453d74d-9d5b-4cf7-bae9-a9ce1c2abea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:12.930289 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 302/612] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:48:12 2022] PUT /identity/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/6b794f7aa2a7488e91f7d9ea35a45d78/roles/ec841a43f9924fa1b2209bc5ecea5e04 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:48:12.931410 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f397ff-0783-4f54-8c3e-f48ca89dd8c4 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:12.978499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:13.041771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 311/613] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:13.061574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f397ff-0783-4f54-8c3e-f48ca89dd8c4 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:13.076677 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453d74d-9d5b-4cf7-bae9-a9ce1c2abea5 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:13.093570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:13.106125 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02efa71-9ab4-431a-b2b4-afb27f4d4cf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:13.106973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02efa71-9ab4-431a-b2b4-afb27f4d4cf8 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:13.107265 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02efa71-9ab4-431a-b2b4-afb27f4d4cf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:13.107548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02efa71-9ab4-431a-b2b4-afb27f4d4cf8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:13.125498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02efa71-9ab4-431a-b2b4-afb27f4d4cf8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:13.126998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02efa71-9ab4-431a-b2b4-afb27f4d4cf8 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:13.158406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-c02efa71-9ab4-431a-b2b4-afb27f4d4cf8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:13.158406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:13.158406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:13.158406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:13.158406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:13.158406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:13.158406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:13.158406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:13.158406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:13.158406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:13.158406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:13.158406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:13.158406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:13.158406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:13.158406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:13.159403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:13.159403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:13.159403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:13.159403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:13.159403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:13.159403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:13.159403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:13.159403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:13.159403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:13.159403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:13.159403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:13.159403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:13.159403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:13.159403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:13.159403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:13.159403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:13.159403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:13.160325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:13.160325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:13.160325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:13.160325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:13.164148 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 303/614] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:13.166785 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02efa71-9ab4-431a-b2b4-afb27f4d4cf8 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:13.197912 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453d74d-9d5b-4cf7-bae9-a9ce1c2abea5 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:13.251935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a579e-fcf6-45cd-ad65-c38bb7f8d0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:13.252771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a579e-fcf6-45cd-ad65-c38bb7f8d0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:13.253062 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a579e-fcf6-45cd-ad65-c38bb7f8d0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:13.253348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681a579e-fcf6-45cd-ad65-c38bb7f8d0b2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:13.296666 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681a579e-fcf6-45cd-ad65-c38bb7f8d0b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:13.305127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-681a579e-fcf6-45cd-ad65-c38bb7f8d0b2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:13.305127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:13.305127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:13.305127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:13.305127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:13.305127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:13.305127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:13.305127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:13.305127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:13.305127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:13.305127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:13.305127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:13.305127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:13.305127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:13.305127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:13.305127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:13.306054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:13.306054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:13.306054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:13.306054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:13.306054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:13.306054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:13.306054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:13.306054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:13.306054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:13.306054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:13.306054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:13.306054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:13.306054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:13.306054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:13.306054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:13.306054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:13.306944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:13.306944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:13.306944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:13.306944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:13.306944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:13.306944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:13.306944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:13.306944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:13.306944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:13.306944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:13.306944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:13.306944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:13.308848 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:13.311091 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 312/615] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:13.316473 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a579e-fcf6-45cd-ad65-c38bb7f8d0b2 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:13.324031 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02efa71-9ab4-431a-b2b4-afb27f4d4cf8 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:13.363468 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfd7aa-8571-4100-9d3a-2a297e3530c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:13.364647 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfd7aa-8571-4100-9d3a-2a297e3530c0 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:13.365735 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfd7aa-8571-4100-9d3a-2a297e3530c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:13.366218 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bcfd7aa-8571-4100-9d3a-2a297e3530c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:13.367318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfd7aa-8571-4100-9d3a-2a297e3530c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:13.367885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfd7aa-8571-4100-9d3a-2a297e3530c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:13.369445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bcfd7aa-8571-4100-9d3a-2a297e3530c0 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:13.382880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 304/616] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:13.392160 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfd7aa-8571-4100-9d3a-2a297e3530c0 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:13.473309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:13.540699 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681a579e-fcf6-45cd-ad65-c38bb7f8d0b2 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:13.580614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bcfd7aa-8571-4100-9d3a-2a297e3530c0 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:13.607502 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e67d5-6e17-4674-baa5-2477a52750c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:13.608265 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e67d5-6e17-4674-baa5-2477a52750c0 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:13.608552 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e67d5-6e17-4674-baa5-2477a52750c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:13.608829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59e67d5-6e17-4674-baa5-2477a52750c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:13.609583 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e67d5-6e17-4674-baa5-2477a52750c0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:13.609893 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e67d5-6e17-4674-baa5-2477a52750c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:13.611555 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59e67d5-6e17-4674-baa5-2477a52750c0 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:13.619097 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600e5e8-3a93-4059-a5f3-5a870eece574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:13.619899 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600e5e8-3a93-4059-a5f3-5a870eece574 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:13.620185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600e5e8-3a93-4059-a5f3-5a870eece574 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:13.620463 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c600e5e8-3a93-4059-a5f3-5a870eece574 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:13.628939 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 313/617] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:48:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:13.634866 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e67d5-6e17-4674-baa5-2477a52750c0 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:13.636643 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600e5e8-3a93-4059-a5f3-5a870eece574 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:13.638104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c600e5e8-3a93-4059-a5f3-5a870eece574 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:13.663595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-c600e5e8-3a93-4059-a5f3-5a870eece574 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 22:48:13.663595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:13.663595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:13.663595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:13.663595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:13.663595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:13.663595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:13.663595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:13.663595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:13.663595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:13.663595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:13.663595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:13.663595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:13.663595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:13.663595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:13.663595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:13.664611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:13.664611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:13.664611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:13.664611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:13.664611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:13.664611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:13.664611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:13.664611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:13.664611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:13.664611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:13.664611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:13.664611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:13.664611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:13.664611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:13.664611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:13.664611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:13.664611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:13.664611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:13.665545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:13.665545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:13.665545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:13.665545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:13.665545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:13.665545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:13.665545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:13.665545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:13.665545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 22:48:13.665545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:13.665545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 305/618] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:48:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:13.670557 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600e5e8-3a93-4059-a5f3-5a870eece574 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:13.688996 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:13.832627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59e67d5-6e17-4674-baa5-2477a52750c0 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:13.860480 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c600e5e8-3a93-4059-a5f3-5a870eece574 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:13.887343 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:13.897754 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659a2bf-699c-4393-b812-46dbfdba2f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:13.899910 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659a2bf-699c-4393-b812-46dbfdba2f7f None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:13.900584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659a2bf-699c-4393-b812-46dbfdba2f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:13.901738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659a2bf-699c-4393-b812-46dbfdba2f7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:13.917603 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659a2bf-699c-4393-b812-46dbfdba2f7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:13.919506 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659a2bf-699c-4393-b812-46dbfdba2f7f None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:13.927409 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ef464-fa1f-423c-994f-a13c3c260ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:13.928655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ef464-fa1f-423c-994f-a13c3c260ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:13.928941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ef464-fa1f-423c-994f-a13c3c260ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:13.929215 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385ef464-fa1f-423c-994f-a13c3c260ef6 None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:13.930047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ef464-fa1f-423c-994f-a13c3c260ef6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:13.930379 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ef464-fa1f-423c-994f-a13c3c260ef6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:13.938613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385ef464-fa1f-423c-994f-a13c3c260ef6 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:13.952479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-9659a2bf-699c-4393-b812-46dbfdba2f7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:13.952479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:13.952479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:13.952479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:13.952479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:13.952479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:13.952479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:13.952479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:13.952479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:13.952479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:13.952479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:13.952479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:13.952479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:13.952479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:13.952479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:13.953549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:13.953549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:13.953549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:13.953549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:13.953549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:13.953549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:13.953549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:13.953549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:13.953549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:13.953549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:13.953549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:13.953549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:13.953549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:13.953549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:13.953549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:13.953549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:13.953549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:13.954420 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:13.954420 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:13.954420 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:13.954420 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:13.954688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 306/619] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jun 16 22:48:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:13.954762 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 314/620] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:13.956234 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ef464-fa1f-423c-994f-a13c3c260ef6 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:13.957752 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659a2bf-699c-4393-b812-46dbfdba2f7f None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:14.048023 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:14.110710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385ef464-fa1f-423c-994f-a13c3c260ef6 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:14.180796 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce180c-382d-4fd2-b81c-64addc3b4c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:14.181603 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce180c-382d-4fd2-b81c-64addc3b4c11 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:14.181890 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce180c-382d-4fd2-b81c-64addc3b4c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:14.182171 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbce180c-382d-4fd2-b81c-64addc3b4c11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:14.184524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659a2bf-699c-4393-b812-46dbfdba2f7f None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:14.205256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce180c-382d-4fd2-b81c-64addc3b4c11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:14.207192 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbce180c-382d-4fd2-b81c-64addc3b4c11 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:14.236810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-cbce180c-382d-4fd2-b81c-64addc3b4c11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:14.236810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:14.236810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:14.236810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:14.236810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:14.236810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:14.236810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:14.236810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:14.236810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:14.236810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:14.236810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:14.236810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:14.236810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:14.236810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:14.236810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:14.237680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:14.237680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:14.237680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:14.237680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:14.237680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:14.237680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:14.237680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:14.237680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:14.237680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:14.237680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:14.237680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:14.237680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:14.237680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:14.237680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:14.237680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:14.237680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:14.237680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:14.238543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:14.238543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:14.238543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:14.238543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:14.252272 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 307/621] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:14.253135 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce180c-382d-4fd2-b81c-64addc3b4c11 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:14.263859 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b962a3b-b58c-466a-872d-e780fa937270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:14.264540 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b962a3b-b58c-466a-872d-e780fa937270 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:14.264825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b962a3b-b58c-466a-872d-e780fa937270 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:14.265099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b962a3b-b58c-466a-872d-e780fa937270 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:14.265830 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b962a3b-b58c-466a-872d-e780fa937270 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:14.266138 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b962a3b-b58c-466a-872d-e780fa937270 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:14.267610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b962a3b-b58c-466a-872d-e780fa937270 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:14.288089 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 315/622] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:14.291497 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b962a3b-b58c-466a-872d-e780fa937270 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:14.341353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:14.402369 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b962a3b-b58c-466a-872d-e780fa937270 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:14.445625 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d48df6-132b-4c1b-a5b1-1cb7d2ed8853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:14.446464 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d48df6-132b-4c1b-a5b1-1cb7d2ed8853 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:14.446785 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d48df6-132b-4c1b-a5b1-1cb7d2ed8853 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:14.447070 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d48df6-132b-4c1b-a5b1-1cb7d2ed8853 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:14.454645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d48df6-132b-4c1b-a5b1-1cb7d2ed8853 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:14.456100 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d48df6-132b-4c1b-a5b1-1cb7d2ed8853 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:14.462686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbce180c-382d-4fd2-b81c-64addc3b4c11 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:14.471829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-28d48df6-132b-4c1b-a5b1-1cb7d2ed8853 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 22:48:14.471829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:14.471829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:14.471829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:14.471829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:14.471829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:14.471829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:14.471829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:14.471829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:14.471829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:14.471829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:14.471829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:14.471829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:14.471829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:14.471829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:14.471829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:14.473904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:14.473904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:14.473904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:14.473904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:14.473904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:14.473904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:14.473904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:14.473904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:14.473904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:14.473904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:14.473904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:14.473904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:14.473904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:14.473904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:14.473904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:14.473904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:14.473904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:14.473904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:14.475728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:14.475728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:14.475728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:14.475728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:14.475728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:14.475728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:14.475728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:14.475728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:14.475728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 22:48:14.475728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:14.475728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 316/623] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jun 16 22:48:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:14.482584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d48df6-132b-4c1b-a5b1-1cb7d2ed8853 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:14.497540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:14.533288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa554a-97ac-45c6-b454-660124fb83c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:14.533973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa554a-97ac-45c6-b454-660124fb83c4 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:14.534264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa554a-97ac-45c6-b454-660124fb83c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:14.534548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fa554a-97ac-45c6-b454-660124fb83c4 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:14.535304 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa554a-97ac-45c6-b454-660124fb83c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:14.535617 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa554a-97ac-45c6-b454-660124fb83c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:14.537512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fa554a-97ac-45c6-b454-660124fb83c4 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:14.557483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 308/624] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:14.571979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa554a-97ac-45c6-b454-660124fb83c4 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:14.648887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d48df6-132b-4c1b-a5b1-1cb7d2ed8853 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:14.651243 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:14.725871 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9560a965-545d-49d5-8374-cc9e5f46d9ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:14.726483 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560a965-545d-49d5-8374-cc9e5f46d9ca None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:14.726791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560a965-545d-49d5-8374-cc9e5f46d9ca None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:14.727064 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9560a965-545d-49d5-8374-cc9e5f46d9ca None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:14.727904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560a965-545d-49d5-8374-cc9e5f46d9ca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:14.728214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560a965-545d-49d5-8374-cc9e5f46d9ca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:14.729632 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9560a965-545d-49d5-8374-cc9e5f46d9ca None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:14.757968 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fa554a-97ac-45c6-b454-660124fb83c4 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:14.762166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 317/625] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jun 16 22:48:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:14.769046 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd8684-8fe9-4bd0-96d3-66a1c1b17d62 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:14.769046 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd8684-8fe9-4bd0-96d3-66a1c1b17d62 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:14.769046 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1fd8684-8fe9-4bd0-96d3-66a1c1b17d62 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:14.769046 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 318/626] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:14.769046 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd8684-8fe9-4bd0-96d3-66a1c1b17d62 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:14.806066 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fac5aa-7eb1-47fd-9762-3a30f151530f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:14.808437 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fac5aa-7eb1-47fd-9762-3a30f151530f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:14.808995 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fac5aa-7eb1-47fd-9762-3a30f151530f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:14.809483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fac5aa-7eb1-47fd-9762-3a30f151530f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:14.819435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fac5aa-7eb1-47fd-9762-3a30f151530f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:14.822044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fac5aa-7eb1-47fd-9762-3a30f151530f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:14.842492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-a0fac5aa-7eb1-47fd-9762-3a30f151530f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:14.842492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:14.842492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:14.842492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:14.842492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:14.842492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:14.842492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:14.842492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:14.842492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:14.842492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:14.842492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:14.842492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:14.842492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:14.842492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:14.842492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:14.843467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:14.843467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:14.843467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:14.843467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:14.843467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:14.843467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:14.843467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:14.843467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:14.843467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:14.843467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:14.843467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:14.843467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:14.843467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:14.843467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:14.843467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:14.843467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:14.843467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:14.844337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:14.844337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:14.844337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:14.844337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:14.844337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:14.844337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:14.844337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:14.844337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:14.847638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 309/627] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:14.850120 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c50cd0-4cb3-4fd3-bd1f-04f67885a3f3 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:14.850669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c50cd0-4cb3-4fd3-bd1f-04f67885a3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:14.851172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c50cd0-4cb3-4fd3-bd1f-04f67885a3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:14.869302 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-50c50cd0-4cb3-4fd3-bd1f-04f67885a3f3 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:14.870488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-50c50cd0-4cb3-4fd3-bd1f-04f67885a3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:14.890018 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1fd8684-8fe9-4bd0-96d3-66a1c1b17d62 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:14.903462 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:14.925799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b2dd3-1ef4-4f20-85b9-9cb8e1b076b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:14.926458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b2dd3-1ef4-4f20-85b9-9cb8e1b076b5 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:14.926780 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b2dd3-1ef4-4f20-85b9-9cb8e1b076b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:14.927218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b2dd3-1ef4-4f20-85b9-9cb8e1b076b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:14.949701 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 310/628] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:14.950348 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c50cd0-4cb3-4fd3-bd1f-04f67885a3f3 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:14.967210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:15.012239 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b2dd3-1ef4-4f20-85b9-9cb8e1b076b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:15.014452 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b2dd3-1ef4-4f20-85b9-9cb8e1b076b5 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:15.053693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c50cd0-4cb3-4fd3-bd1f-04f67885a3f3 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:15.096259 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8113a882-0be1-4cdf-a970-992b75435e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:15.097098 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113a882-0be1-4cdf-a970-992b75435e1c None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:15.097391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113a882-0be1-4cdf-a970-992b75435e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:15.097677 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8113a882-0be1-4cdf-a970-992b75435e1c None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:15.098467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8113a882-0be1-4cdf-a970-992b75435e1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:15.098826 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8113a882-0be1-4cdf-a970-992b75435e1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:15.100809 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8113a882-0be1-4cdf-a970-992b75435e1c None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:15.112403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 311/629] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:15.116680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074fb59-2a15-4759-b1df-06bba7ed80c3 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:15.117048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074fb59-2a15-4759-b1df-06bba7ed80c3 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:15.117401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074fb59-2a15-4759-b1df-06bba7ed80c3 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:15.119769 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 312/630] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:15.120556 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074fb59-2a15-4759-b1df-06bba7ed80c3 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:15.126414 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 319/631] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:48:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 22:48:15.127542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b2dd3-1ef4-4f20-85b9-9cb8e1b076b5 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:15.152225 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:15.190476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:15.233285 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b2dd3-1ef4-4f20-85b9-9cb8e1b076b5 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:15.239155 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074fb59-2a15-4759-b1df-06bba7ed80c3 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:15.269754 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1ffa7-10ab-4087-8b19-380c3a4dc5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:15.270577 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1ffa7-10ab-4087-8b19-380c3a4dc5e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:15.270884 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1ffa7-10ab-4087-8b19-380c3a4dc5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:15.271212 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de1ffa7-10ab-4087-8b19-380c3a4dc5e1 None admin] PATH_INFO: `/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/bef518c9430e4274b1066241f821f4a1/roles/ec841a43f9924fa1b2209bc5ecea5e04` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:15.277218 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e97be80-08d0-4fb3-8024-f887f3a945ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:15.278018 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97be80-08d0-4fb3-8024-f887f3a945ac None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:15.278567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97be80-08d0-4fb3-8024-f887f3a945ac None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:15.278893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e97be80-08d0-4fb3-8024-f887f3a945ac None admin] PATH_INFO: `/v3/roles/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:15.286196 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1ffa7-10ab-4087-8b19-380c3a4dc5e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc48914270df4db996bd2a4b5b9fc777, user_id=bef518c9430e4274b1066241f821f4a1, role_id=ec841a43f9924fa1b2209bc5ecea5e04)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:15.288230 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de1ffa7-10ab-4087-8b19-380c3a4dc5e1 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:15.290095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e97be80-08d0-4fb3-8024-f887f3a945ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:15.297858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-0e97be80-08d0-4fb3-8024-f887f3a945ac None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:15.297858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:15.297858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:15.297858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:15.297858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:15.297858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:15.297858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:15.297858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:15.297858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:15.297858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:15.297858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:15.297858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:15.297858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:15.297858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:15.297858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:15.297858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:15.298877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:15.298877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:15.298877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:15.298877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:15.298877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:15.298877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:15.298877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:15.298877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:15.298877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:15.298877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:15.298877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:15.298877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:15.298877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:15.298877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:15.298877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:15.298877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:15.299688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:15.299688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:15.299688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:15.299688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:15.299688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:15.299688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:15.299688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:15.299688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:15.299688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:15.299688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:15.299688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:15.299688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:15.299688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 313/632] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:15.300371 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f25bd-9845-41db-8142-8fea262302af None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:15.301074 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f25bd-9845-41db-8142-8fea262302af None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:15.301074 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813f25bd-9845-41db-8142-8fea262302af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:15.306821 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 320/633] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:48:15 2022] PUT /identity/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/bef518c9430e4274b1066241f821f4a1/roles/ec841a43f9924fa1b2209bc5ecea5e04 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:48:15.307631 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1ffa7-10ab-4087-8b19-380c3a4dc5e1 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:15.333423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-813f25bd-9845-41db-8142-8fea262302af None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:15.335346 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-813f25bd-9845-41db-8142-8fea262302af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:15.416759 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 314/634] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:15.417696 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f25bd-9845-41db-8142-8fea262302af None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:15.440298 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:15.472298 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de1ffa7-10ab-4087-8b19-380c3a4dc5e1 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:15.490158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:15.521928 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ea21c-f456-4b02-ac50-5ab5903e168b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:15.522818 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ea21c-f456-4b02-ac50-5ab5903e168b None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:15.523136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ea21c-f456-4b02-ac50-5ab5903e168b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:15.523418 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229ea21c-f456-4b02-ac50-5ab5903e168b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:15.534915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ea21c-f456-4b02-ac50-5ab5903e168b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:15.536357 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229ea21c-f456-4b02-ac50-5ab5903e168b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:15.540737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813f25bd-9845-41db-8142-8fea262302af None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:15.557646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-229ea21c-f456-4b02-ac50-5ab5903e168b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:15.557646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:15.557646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:15.557646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:15.557646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:15.557646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:15.557646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:15.557646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:15.557646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:15.557646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:15.557646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:15.557646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:15.557646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:15.557646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:15.557646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:15.558579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:15.558579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:15.558579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:15.558579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:15.558579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:15.558579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:15.558579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:15.558579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:15.558579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:15.558579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:15.558579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:15.558579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:15.558579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:15.558579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:15.558579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:15.558579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:15.558579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:15.559521 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:15.559521 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:15.559521 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:15.559521 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:15.559521 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 321/635] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:15.560344 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ea21c-f456-4b02-ac50-5ab5903e168b None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:15.581747 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ad4108-6a93-4188-be76-48f3645c4ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:15.583175 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad4108-6a93-4188-be76-48f3645c4ccf None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:15.584167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad4108-6a93-4188-be76-48f3645c4ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:15.584453 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ad4108-6a93-4188-be76-48f3645c4ccf None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:15.585200 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad4108-6a93-4188-be76-48f3645c4ccf None admin] RBAC: Adding query filter params (name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:15.585509 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad4108-6a93-4188-be76-48f3645c4ccf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:15.586911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ad4108-6a93-4188-be76-48f3645c4ccf None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:15.600248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 315/636] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:48:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:15.600248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136afa0-fbae-43f1-85ce-8e3bf07f7346 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:15.600248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136afa0-fbae-43f1-85ce-8e3bf07f7346 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:15.600248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7136afa0-fbae-43f1-85ce-8e3bf07f7346 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:15.600248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 316/637] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:15 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:15.600248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136afa0-fbae-43f1-85ce-8e3bf07f7346 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:15.674092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229ea21c-f456-4b02-ac50-5ab5903e168b None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:15.685365 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:15.707482 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7136afa0-fbae-43f1-85ce-8e3bf07f7346 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:15.711493 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e8a1d-26fb-4916-b6a2-78dc46ce4f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:15.712490 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e8a1d-26fb-4916-b6a2-78dc46ce4f1f None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:15.712933 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e8a1d-26fb-4916-b6a2-78dc46ce4f1f None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:15.713362 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e8a1d-26fb-4916-b6a2-78dc46ce4f1f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:15.724485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e8a1d-26fb-4916-b6a2-78dc46ce4f1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:15.732189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-079e8a1d-26fb-4916-b6a2-78dc46ce4f1f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:15.732189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:15.732189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:15.732189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:15.732189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:15.732189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:15.732189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:15.732189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:15.732189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:15.732189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:15.732189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:15.732189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:15.732189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:15.732189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:15.732189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:15.732189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:15.733145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:15.733145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:15.733145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:15.733145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:15.733145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:15.733145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:15.733145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:15.733145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:15.733145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:15.733145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:15.733145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:15.733145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:15.733145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:15.733145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:15.733145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:15.733145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:15.733983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:15.733983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:15.733983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:15.733983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:15.733983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:15.733983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:15.733983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:15.733983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:15.733983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:15.733983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:15.733983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:15.733983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:15.733983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 322/638] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:15.734719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e8a1d-26fb-4916-b6a2-78dc46ce4f1f None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:15.747100 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e974b01-7c44-4fc8-a83e-c1138eea7a84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:15.747772 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e974b01-7c44-4fc8-a83e-c1138eea7a84 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:15.748058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e974b01-7c44-4fc8-a83e-c1138eea7a84 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:15.748335 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e974b01-7c44-4fc8-a83e-c1138eea7a84 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:15.749059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e974b01-7c44-4fc8-a83e-c1138eea7a84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:15.749362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e974b01-7c44-4fc8-a83e-c1138eea7a84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:15.750745 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e974b01-7c44-4fc8-a83e-c1138eea7a84 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:15.763168 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 317/639] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:15.764394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cfd0cd-5093-4315-90ca-8ce036b44345 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:15.764683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cfd0cd-5093-4315-90ca-8ce036b44345 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:15.764954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cfd0cd-5093-4315-90ca-8ce036b44345 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:15.769342 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:15.781273 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-f6cfd0cd-5093-4315-90ca-8ce036b44345 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:15.782383 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f6cfd0cd-5093-4315-90ca-8ce036b44345 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:15.853461 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 318/640] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:15.854303 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e8a1d-26fb-4916-b6a2-78dc46ce4f1f None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:15.854741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cfd0cd-5093-4315-90ca-8ce036b44345 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:15.897582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a2645-b2b7-450a-8cc0-f93e54583a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:15.898376 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a2645-b2b7-450a-8cc0-f93e54583a9c None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:15.898688 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a2645-b2b7-450a-8cc0-f93e54583a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:15.898966 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a2645-b2b7-450a-8cc0-f93e54583a9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:15.910441 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a2645-b2b7-450a-8cc0-f93e54583a9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:15.912590 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a2645-b2b7-450a-8cc0-f93e54583a9c None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:15.931065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-5a4a2645-b2b7-450a-8cc0-f93e54583a9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:15.931065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:15.931065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:15.931065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:15.931065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:15.931065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:15.931065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:15.931065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:15.931065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:15.931065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:15.931065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:15.931065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:15.931065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:15.931065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:15.931065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:15.931959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:15.931959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:15.931959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:15.931959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:15.931959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:15.931959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:15.931959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:15.931959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:15.931959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:15.931959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:15.931959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:15.931959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:15.931959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:15.931959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:15.931959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:15.931959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:15.931959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:15.937732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:15.937732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:15.937732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:15.937732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:15.938013 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:15.938096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 323/641] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:15.938096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a2645-b2b7-450a-8cc0-f93e54583a9c None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:15.971575 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cfd0cd-5093-4315-90ca-8ce036b44345 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:16.007877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912fabe-9e5f-4ce1-bff3-ebaed01f91c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:16.008546 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912fabe-9e5f-4ce1-bff3-ebaed01f91c9 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:16.008837 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912fabe-9e5f-4ce1-bff3-ebaed01f91c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:16.009114 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7912fabe-9e5f-4ce1-bff3-ebaed01f91c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:16.009889 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912fabe-9e5f-4ce1-bff3-ebaed01f91c9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:16.010163 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912fabe-9e5f-4ce1-bff3-ebaed01f91c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:16.011559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7912fabe-9e5f-4ce1-bff3-ebaed01f91c9 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:16.019919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 319/642] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:48:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:16.020408 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:16.020602 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912fabe-9e5f-4ce1-bff3-ebaed01f91c9 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:16.050797 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a2645-b2b7-450a-8cc0-f93e54583a9c None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:16.088267 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bc480-a60e-4fe5-842f-beafa2bd9857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:16.089061 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bc480-a60e-4fe5-842f-beafa2bd9857 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:16.089346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bc480-a60e-4fe5-842f-beafa2bd9857 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:16.089620 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122bc480-a60e-4fe5-842f-beafa2bd9857 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:16.096465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bc480-a60e-4fe5-842f-beafa2bd9857 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:16.097899 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122bc480-a60e-4fe5-842f-beafa2bd9857 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:16.112693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-122bc480-a60e-4fe5-842f-beafa2bd9857 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:16.112693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:16.112693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:16.112693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:16.112693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:16.112693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:16.112693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:16.112693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:16.112693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:16.112693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:16.112693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:16.112693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:16.112693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:16.112693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:16.112693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:16.113646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:16.113646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:16.113646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:16.113646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:16.113646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:16.113646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:16.113646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:16.113646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:16.113646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:16.113646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:16.113646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:16.113646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:16.113646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:16.113646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:16.113646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:16.113646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:16.113646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:16.114524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:16.114524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:16.114524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:16.114524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:16.114524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:16.114524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:16.114524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:16.114524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:16.114524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 324/643] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:16.115623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bc480-a60e-4fe5-842f-beafa2bd9857 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:16.132561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7912fabe-9e5f-4ce1-bff3-ebaed01f91c9 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:16.164621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:16.171432 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc4194-55e6-4995-9705-dcce56f05e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:16.172370 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc4194-55e6-4995-9705-dcce56f05e67 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:16.172657 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc4194-55e6-4995-9705-dcce56f05e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:16.172944 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ecc4194-55e6-4995-9705-dcce56f05e67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:16.183220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ecc4194-55e6-4995-9705-dcce56f05e67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:16.191009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-0ecc4194-55e6-4995-9705-dcce56f05e67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:16.191009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:16.191009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:16.191009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:16.191009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:16.191009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:16.191009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:16.191009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:16.191009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:16.191009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:16.191009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:16.191009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:16.191009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:16.191009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:16.191009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:16.191009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:16.191886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:16.191886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:16.191886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:16.191886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:16.191886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:16.191886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:16.191886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:16.191886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:16.191886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:16.191886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:16.191886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:16.191886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:16.191886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:16.191886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:16.191886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:16.191886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:16.192766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:16.192766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:16.192766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:16.192766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:16.192766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:16.192766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:16.192766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:16.192766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:16.192766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:16.192766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:16.192766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:16.192766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:16.194814 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 320/644] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:16.195736 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc4194-55e6-4995-9705-dcce56f05e67 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:16.237126 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122bc480-a60e-4fe5-842f-beafa2bd9857 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:16.272045 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6db8f-0b67-4b8f-b760-41d646cd600c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:16.272750 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6db8f-0b67-4b8f-b760-41d646cd600c None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:16.273038 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6db8f-0b67-4b8f-b760-41d646cd600c None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:16.273320 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf6db8f-0b67-4b8f-b760-41d646cd600c None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:16.274021 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6db8f-0b67-4b8f-b760-41d646cd600c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:16.274339 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6db8f-0b67-4b8f-b760-41d646cd600c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:16.275728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf6db8f-0b67-4b8f-b760-41d646cd600c None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:16.277259 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:16.292724 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 325/645] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:16.292724 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6db8f-0b67-4b8f-b760-41d646cd600c None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:16.311105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ecc4194-55e6-4995-9705-dcce56f05e67 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:16.345841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af2b60-349b-437a-a638-ae1c5fcccae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:16.346779 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af2b60-349b-437a-a638-ae1c5fcccae1 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:16.347068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af2b60-349b-437a-a638-ae1c5fcccae1 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:16.347463 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af2b60-349b-437a-a638-ae1c5fcccae1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:16.359600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af2b60-349b-437a-a638-ae1c5fcccae1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:16.361170 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29af2b60-349b-437a-a638-ae1c5fcccae1 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:16.381954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-29af2b60-349b-437a-a638-ae1c5fcccae1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:16.381954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:16.381954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:16.381954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:16.381954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:16.381954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:16.381954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:16.381954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:16.381954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:16.381954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:16.381954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:16.381954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:16.381954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:16.381954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:16.381954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:16.383025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:16.383025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:16.383025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:16.383025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:16.383025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:16.383025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:16.383025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:16.383025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:16.383025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:16.383025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:16.383025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:16.383025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:16.383025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:16.383025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:16.383025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:16.383025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:16.383025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:16.384081 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:16.384081 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:16.384081 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:16.384081 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:16.386863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 321/646] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:16.387720 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af2b60-349b-437a-a638-ae1c5fcccae1 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:16.397669 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf6db8f-0b67-4b8f-b760-41d646cd600c None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:16.434762 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90612a76-1f4d-48c9-8967-6dd7796b4ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:16.435433 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90612a76-1f4d-48c9-8967-6dd7796b4ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:16.435720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90612a76-1f4d-48c9-8967-6dd7796b4ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:16.435996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90612a76-1f4d-48c9-8967-6dd7796b4ab9 None admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:16.436793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90612a76-1f4d-48c9-8967-6dd7796b4ab9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:16.437101 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90612a76-1f4d-48c9-8967-6dd7796b4ab9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:16.438529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90612a76-1f4d-48c9-8967-6dd7796b4ab9 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:16.441512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:16.451398 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 326/647] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:16.452082 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90612a76-1f4d-48c9-8967-6dd7796b4ab9 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:16.508527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29af2b60-349b-437a-a638-ae1c5fcccae1 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:16.512481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:16.549816 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad30a7-1ac8-469b-84bf-52d67ee46630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:16.550493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad30a7-1ac8-469b-84bf-52d67ee46630 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:16.550812 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad30a7-1ac8-469b-84bf-52d67ee46630 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:16.551090 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ad30a7-1ac8-469b-84bf-52d67ee46630 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:16.551827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad30a7-1ac8-469b-84bf-52d67ee46630 None admin] RBAC: Adding query filter params (name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:16.552135 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad30a7-1ac8-469b-84bf-52d67ee46630 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:16.554218 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ad30a7-1ac8-469b-84bf-52d67ee46630 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:16.558785 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90612a76-1f4d-48c9-8967-6dd7796b4ab9 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:16.566878 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 322/648] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:48:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:16.566878 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad30a7-1ac8-469b-84bf-52d67ee46630 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:16.596499 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130fd6f-cac3-40ce-b68e-b96a300261d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:16.597299 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130fd6f-cac3-40ce-b68e-b96a300261d6 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:16.597584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130fd6f-cac3-40ce-b68e-b96a300261d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:16.597881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d130fd6f-cac3-40ce-b68e-b96a300261d6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:16.605524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d130fd6f-cac3-40ce-b68e-b96a300261d6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:16.606976 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d130fd6f-cac3-40ce-b68e-b96a300261d6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:16.621768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-d130fd6f-cac3-40ce-b68e-b96a300261d6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 22:48:16.621768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:16.621768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:16.621768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:16.621768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:16.621768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:16.621768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:16.621768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:16.621768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:16.621768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:16.621768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:16.621768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:16.621768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:16.621768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:16.621768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:16.621768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:16.622991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:16.622991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:16.622991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:16.622991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:16.622991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:16.622991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:16.622991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:16.622991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:16.622991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:16.622991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:16.622991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:16.622991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:16.622991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:16.622991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:16.622991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:16.622991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:16.622991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:16.622991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:16.624108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:16.624108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:16.624108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:16.624108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:16.624108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:16.624108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:16.624108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:16.624108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:16.624108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 22:48:16.624108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:16.627658 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 327/649] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:48:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:16.628176 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130fd6f-cac3-40ce-b68e-b96a300261d6 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:16.674885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ad30a7-1ac8-469b-84bf-52d67ee46630 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:16.682458 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:16.711318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29749897-c3d0-490f-b2e2-93f74e4b6c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:16.711979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29749897-c3d0-490f-b2e2-93f74e4b6c2a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:16.712264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29749897-c3d0-490f-b2e2-93f74e4b6c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:16.712543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29749897-c3d0-490f-b2e2-93f74e4b6c2a None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:16.713258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29749897-c3d0-490f-b2e2-93f74e4b6c2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:16.713564 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29749897-c3d0-490f-b2e2-93f74e4b6c2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:16.714974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29749897-c3d0-490f-b2e2-93f74e4b6c2a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:16.726948 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 323/650] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:16.727717 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29749897-c3d0-490f-b2e2-93f74e4b6c2a None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:16.744050 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d130fd6f-cac3-40ce-b68e-b96a300261d6 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:16.761670 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:16.783129 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4cb78-5322-403b-9abb-2d4b2b4317c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:16.783783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4cb78-5322-403b-9abb-2d4b2b4317c0 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:16.784068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4cb78-5322-403b-9abb-2d4b2b4317c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:16.784346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a4cb78-5322-403b-9abb-2d4b2b4317c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:16.785166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4cb78-5322-403b-9abb-2d4b2b4317c0 None admin] RBAC: Adding query filter params (role.id=ec841a43f9924fa1b2209bc5ecea5e04, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=4180f2593149437fa27effd8d1338d10) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:16.785717 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4cb78-5322-403b-9abb-2d4b2b4317c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:16.787436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a4cb78-5322-403b-9abb-2d4b2b4317c0 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:16.793517 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 328/651] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:48:16 2022] GET /identity/v3/role_assignments?user.id=4180f2593149437fa27effd8d1338d10&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=ec841a43f9924fa1b2209bc5ecea5e04 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:16.794283 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4cb78-5322-403b-9abb-2d4b2b4317c0 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:16.852827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29749897-c3d0-490f-b2e2-93f74e4b6c2a None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:16.896192 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc837b-5fa5-49f9-b36d-e9c089fc2ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:16.897760 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc837b-5fa5-49f9-b36d-e9c089fc2ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:16.898124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc837b-5fa5-49f9-b36d-e9c089fc2ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:16.898435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befc837b-5fa5-49f9-b36d-e9c089fc2ec4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:16.910234 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc837b-5fa5-49f9-b36d-e9c089fc2ec4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:16.912151 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befc837b-5fa5-49f9-b36d-e9c089fc2ec4 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:16.926048 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a4cb78-5322-403b-9abb-2d4b2b4317c0 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:16.934645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-befc837b-5fa5-49f9-b36d-e9c089fc2ec4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:16.934645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:16.934645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:16.934645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:16.934645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:16.934645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:16.934645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:16.934645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:16.934645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:16.934645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:16.934645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:16.934645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:16.934645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:16.934645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:16.934645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:16.935798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:16.935798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:16.935798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:16.935798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:16.935798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:16.935798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:16.935798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:16.935798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:16.935798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:16.935798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:16.935798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:16.935798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:16.935798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:16.935798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:16.935798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:16.935798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:16.935798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:16.936888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:16.936888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:16.936888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:16.936888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:16.942608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 324/652] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:16.942608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc837b-5fa5-49f9-b36d-e9c089fc2ec4 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:16.967929 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:16.981934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867adc07-5768-4580-bc64-559eb6fab05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:16.982666 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867adc07-5768-4580-bc64-559eb6fab05d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:16.983077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867adc07-5768-4580-bc64-559eb6fab05d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:16.983377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867adc07-5768-4580-bc64-559eb6fab05d None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:16.984228 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867adc07-5768-4580-bc64-559eb6fab05d None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:16.984980 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867adc07-5768-4580-bc64-559eb6fab05d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:16.986304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867adc07-5768-4580-bc64-559eb6fab05d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:17.008827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 329/653] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jun 16 22:48:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:17.009742 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867adc07-5768-4580-bc64-559eb6fab05d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:17.051732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:17.075208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befc837b-5fa5-49f9-b36d-e9c089fc2ec4 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:17.125479 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5655a25-6560-4e69-b88e-01dcf0a15232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:17.127269 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5655a25-6560-4e69-b88e-01dcf0a15232 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:17.127839 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5655a25-6560-4e69-b88e-01dcf0a15232 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:17.128539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5655a25-6560-4e69-b88e-01dcf0a15232 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:17.132402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867adc07-5768-4580-bc64-559eb6fab05d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:17.139182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5655a25-6560-4e69-b88e-01dcf0a15232 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:17.141540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5655a25-6560-4e69-b88e-01dcf0a15232 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:17.161087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-a5655a25-6560-4e69-b88e-01dcf0a15232 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 22:48:17.161087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:17.161087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:17.161087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:17.161087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:17.161087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:17.161087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:17.161087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:17.161087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:17.161087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:17.161087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:17.161087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:17.161087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:17.161087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:17.161087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:17.161087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:17.162087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:17.162087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:17.162087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:17.162087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:17.162087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:17.162087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:17.162087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:17.162087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:17.162087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:17.162087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:17.162087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:17.162087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:17.162087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:17.162087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:17.162087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:17.162087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:17.162087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:17.162087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:17.163068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:17.163068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:17.163068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:17.163068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:17.163068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:17.163068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:17.163068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:17.163068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:17.163068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 22:48:17.163068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:17.163068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 325/654] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:48:16 2022] GET /identity/v3/users/cinder => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:17.163652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5655a25-6560-4e69-b88e-01dcf0a15232 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:17.179691 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba951c2-9d2c-4c19-abfe-9d0045a78451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:17.181073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba951c2-9d2c-4c19-abfe-9d0045a78451 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:17.181809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba951c2-9d2c-4c19-abfe-9d0045a78451 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:17.182273 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba951c2-9d2c-4c19-abfe-9d0045a78451 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:17.183457 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba951c2-9d2c-4c19-abfe-9d0045a78451 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:17.184409 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba951c2-9d2c-4c19-abfe-9d0045a78451 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:17.186263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba951c2-9d2c-4c19-abfe-9d0045a78451 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:17.202730 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 330/655] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:17.204943 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba951c2-9d2c-4c19-abfe-9d0045a78451 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:17.260217 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:17.284487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5655a25-6560-4e69-b88e-01dcf0a15232 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:17.327268 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f87e2-6e17-4f9b-9847-9d2ca6be3035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:17.328767 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f87e2-6e17-4f9b-9847-9d2ca6be3035 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:17.329500 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f87e2-6e17-4f9b-9847-9d2ca6be3035 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:17.329973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854f87e2-6e17-4f9b-9847-9d2ca6be3035 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:17.335099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:17.344136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba951c2-9d2c-4c19-abfe-9d0045a78451 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:17.381232 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315114cd-7fdb-4107-bb5d-fb2a977efef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:17.381878 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315114cd-7fdb-4107-bb5d-fb2a977efef6 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:17.382169 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315114cd-7fdb-4107-bb5d-fb2a977efef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:17.382517 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315114cd-7fdb-4107-bb5d-fb2a977efef6 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:17.383352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315114cd-7fdb-4107-bb5d-fb2a977efef6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:17.384361 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315114cd-7fdb-4107-bb5d-fb2a977efef6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:17.385095 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315114cd-7fdb-4107-bb5d-fb2a977efef6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:17.411251 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 331/656] 10.222.0.17 () {56 vars in 1137 bytes} [Thu Jun 16 22:48:17 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:17.413190 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315114cd-7fdb-4107-bb5d-fb2a977efef6 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:17.425642 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f87e2-6e17-4f9b-9847-9d2ca6be3035 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:17.428227 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854f87e2-6e17-4f9b-9847-9d2ca6be3035 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:17.523424 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:17.534734 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315114cd-7fdb-4107-bb5d-fb2a977efef6 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:17.552075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 326/657] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:48:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:48:17.553524 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f87e2-6e17-4f9b-9847-9d2ca6be3035 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:17.572989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aa2b86-e2fe-4dd2-a0dc-8099050b66a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:17.573789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa2b86-e2fe-4dd2-a0dc-8099050b66a6 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:17.574073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa2b86-e2fe-4dd2-a0dc-8099050b66a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:17.574347 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aa2b86-e2fe-4dd2-a0dc-8099050b66a6 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:17.581708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa2b86-e2fe-4dd2-a0dc-8099050b66a6 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:17.583369 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aa2b86-e2fe-4dd2-a0dc-8099050b66a6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:17.598571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-31aa2b86-e2fe-4dd2-a0dc-8099050b66a6 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 22:48:17.598571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:17.598571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:17.598571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:17.598571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:17.598571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:17.598571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:17.598571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:17.598571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:17.598571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:17.598571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:17.598571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:17.598571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:17.598571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:17.598571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:17.598571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:17.600078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:17.600078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:17.600078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:17.600078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:17.600078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:17.600078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:17.600078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:17.600078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:17.600078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:17.600078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:17.600078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:17.600078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:17.600078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:17.600078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:17.600078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:17.600078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:17.600078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:17.600078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:17.601043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:17.601043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:17.601043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:17.601043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:17.601043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:17.601043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:17.601043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:17.601043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:17.601043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 22:48:17.601043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:17.601043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 332/658] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:17 2022] GET /identity/v3/users/neutron => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:17.601634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff980ace-ab8e-4aa7-8db2-5b0d501656d0 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:17.602326 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff980ace-ab8e-4aa7-8db2-5b0d501656d0 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:17.602326 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff980ace-ab8e-4aa7-8db2-5b0d501656d0 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:17.603757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 333/659] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:17.604527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff980ace-ab8e-4aa7-8db2-5b0d501656d0 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:17.621915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:17.667124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854f87e2-6e17-4f9b-9847-9d2ca6be3035 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:17.703886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9aa6a9-31fa-44e8-bacc-8bf281b0fea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:17.704536 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9aa6a9-31fa-44e8-bacc-8bf281b0fea7 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:17.704818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9aa6a9-31fa-44e8-bacc-8bf281b0fea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:17.705091 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9aa6a9-31fa-44e8-bacc-8bf281b0fea7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:17.712887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff980ace-ab8e-4aa7-8db2-5b0d501656d0 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:17.748265 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ab273b-f902-4b83-a5cb-ec2e1e8f2574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:17.749068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab273b-f902-4b83-a5cb-ec2e1e8f2574 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:17.749353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab273b-f902-4b83-a5cb-ec2e1e8f2574 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:17.749639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ab273b-f902-4b83-a5cb-ec2e1e8f2574 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:17.753124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:17.760890 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab273b-f902-4b83-a5cb-ec2e1e8f2574 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:17.762330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ab273b-f902-4b83-a5cb-ec2e1e8f2574 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:17.778843 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9aa6a9-31fa-44e8-bacc-8bf281b0fea7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:17.780286 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9aa6a9-31fa-44e8-bacc-8bf281b0fea7 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:17.780469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-84ab273b-f902-4b83-a5cb-ec2e1e8f2574 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:17.780469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:17.780469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:17.780469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:17.780469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:17.780469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:17.780469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:17.780469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:17.780469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:17.780469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:17.780469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:17.780469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:17.780469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:17.780469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:17.780469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:17.781297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:17.781297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:17.781297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:17.781297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:17.781297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:17.781297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:17.781297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:17.781297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:17.781297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:17.781297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:17.781297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:17.781297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:17.781297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:17.781297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:17.781297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:17.781297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:17.781297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:17.782157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:17.782157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:17.782157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:17.782157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:17.786236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 334/660] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:17.787576 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861cac9-8a27-46dc-8e55-c775d5531779 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:17.787865 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861cac9-8a27-46dc-8e55-c775d5531779 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:17.788141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1861cac9-8a27-46dc-8e55-c775d5531779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:17.804786 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-1861cac9-8a27-46dc-8e55-c775d5531779 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:17.805959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-1861cac9-8a27-46dc-8e55-c775d5531779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:17.867591 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:17.876733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 335/661] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:17.877520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861cac9-8a27-46dc-8e55-c775d5531779 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:17.887357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 327/662] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:48:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:48:17.888685 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9aa6a9-31fa-44e8-bacc-8bf281b0fea7 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:17.972113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:17.995502 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1861cac9-8a27-46dc-8e55-c775d5531779 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:17.997029 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9aa6a9-31fa-44e8-bacc-8bf281b0fea7 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:18.031950 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f258cd-83d5-44f4-9291-f6ef41cecb2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:18.032638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f258cd-83d5-44f4-9291-f6ef41cecb2c None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:18.032910 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f258cd-83d5-44f4-9291-f6ef41cecb2c None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:18.033192 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f258cd-83d5-44f4-9291-f6ef41cecb2c None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:18.033929 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f258cd-83d5-44f4-9291-f6ef41cecb2c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:18.034238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f258cd-83d5-44f4-9291-f6ef41cecb2c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:18.034686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e999e7-8264-4539-bf25-ca5094eb6e74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:18.035607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e999e7-8264-4539-bf25-ca5094eb6e74 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:18.035729 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f258cd-83d5-44f4-9291-f6ef41cecb2c None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:18.036178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e999e7-8264-4539-bf25-ca5094eb6e74 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:18.036598 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e999e7-8264-4539-bf25-ca5094eb6e74 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:18.038293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e999e7-8264-4539-bf25-ca5094eb6e74 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:18.038898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e999e7-8264-4539-bf25-ca5094eb6e74 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:18.040397 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e999e7-8264-4539-bf25-ca5094eb6e74 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:18.053454 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 328/663] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:18.053454 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f258cd-83d5-44f4-9291-f6ef41cecb2c None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:18.068685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 336/664] 10.222.0.17 () {56 vars in 1139 bytes} [Thu Jun 16 22:48:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:18.068685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e999e7-8264-4539-bf25-ca5094eb6e74 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:18.123172 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:18.160651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f258cd-83d5-44f4-9291-f6ef41cecb2c None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:18.171209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e999e7-8264-4539-bf25-ca5094eb6e74 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:18.196129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b87b20-af7a-426c-af20-23493363793e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:18.197086 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b87b20-af7a-426c-af20-23493363793e None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:18.197370 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b87b20-af7a-426c-af20-23493363793e None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:18.197715 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b87b20-af7a-426c-af20-23493363793e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:18.204708 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:18.205427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01ed9f-d1ac-46e6-96c7-02893456c1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:18.206333 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01ed9f-d1ac-46e6-96c7-02893456c1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:18.208454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01ed9f-d1ac-46e6-96c7-02893456c1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:18.209697 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c01ed9f-d1ac-46e6-96c7-02893456c1e8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:18.212496 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b87b20-af7a-426c-af20-23493363793e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:18.214130 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b87b20-af7a-426c-af20-23493363793e None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:18.221524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c01ed9f-d1ac-46e6-96c7-02893456c1e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:18.231112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-0c01ed9f-d1ac-46e6-96c7-02893456c1e8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:18.231112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:18.231112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:18.231112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:18.231112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:18.231112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:18.231112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:18.231112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:18.231112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:18.231112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:18.231112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:18.231112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:18.231112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:18.231112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:18.231112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:18.231112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:18.236945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:18.236945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:18.236945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:18.236945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:18.236945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:18.236945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:18.236945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:18.236945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:18.236945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:18.236945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:18.236945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:18.236945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:18.236945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:18.236945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:18.236945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:18.236945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:18.237978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:18.237978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:18.237978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:18.237978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:18.237978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:18.237978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:18.237978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:18.237978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:18.237978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:18.237978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:18.237978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:18.237978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:18.237978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 337/665] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:18.238788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-54b87b20-af7a-426c-af20-23493363793e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:18.238788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:18.238788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:18.238788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:18.238788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:18.238788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:18.238788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:18.238788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:18.238788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:18.238788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:18.238788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:18.238788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:18.238788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:18.238788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:18.238788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:18.239751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:18.239751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:18.239751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:18.239751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:18.239751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:18.239751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:18.239751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:18.239751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:18.239751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:18.239751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:18.239751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:18.239751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:18.239751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:18.239751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:18.239751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:18.239751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:18.239751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:18.242534 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:18.242534 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:18.242534 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:18.242534 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:18.242844 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01ed9f-d1ac-46e6-96c7-02893456c1e8 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:18.242928 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 329/666] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:18.242928 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b87b20-af7a-426c-af20-23493363793e None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:18.341746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c01ed9f-d1ac-46e6-96c7-02893456c1e8 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:18.346359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b87b20-af7a-426c-af20-23493363793e None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:18.355508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:18.377309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc71b1d-7911-46d2-9446-99e08958867b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:18.378102 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc71b1d-7911-46d2-9446-99e08958867b None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:18.378384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc71b1d-7911-46d2-9446-99e08958867b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:18.378688 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc71b1d-7911-46d2-9446-99e08958867b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:18.383655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23decae3-8b00-4de2-99bf-a5fcda20a86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:18.384311 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23decae3-8b00-4de2-99bf-a5fcda20a86a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:18.384600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23decae3-8b00-4de2-99bf-a5fcda20a86a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:18.385104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23decae3-8b00-4de2-99bf-a5fcda20a86a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:18.385718 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc71b1d-7911-46d2-9446-99e08958867b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:18.387175 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc71b1d-7911-46d2-9446-99e08958867b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:18.402134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-5bc71b1d-7911-46d2-9446-99e08958867b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:18.402134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:18.402134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:18.402134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:18.402134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:18.402134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:18.402134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:18.402134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:18.402134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:18.402134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:18.402134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:18.402134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:18.402134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:18.402134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:18.402134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:18.403461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:18.403461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:18.403461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:18.403461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:18.403461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:18.403461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:18.403461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:18.403461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:18.403461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:18.403461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:18.403461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:18.403461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:18.403461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:18.403461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:18.403461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:18.403461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:18.403461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:18.404348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:18.404348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:18.404348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:18.404348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:18.404348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:18.404348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:18.404348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:18.404348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:18.404348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 338/667] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:18.404815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc71b1d-7911-46d2-9446-99e08958867b None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:18.451901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:18.464265 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23decae3-8b00-4de2-99bf-a5fcda20a86a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:18.465833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23decae3-8b00-4de2-99bf-a5fcda20a86a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:18.519558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc71b1d-7911-46d2-9446-99e08958867b None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:18.555855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3b119-9c9e-4a20-b28e-cf7a9d5e2110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:18.556527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3b119-9c9e-4a20-b28e-cf7a9d5e2110 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:18.556816 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3b119-9c9e-4a20-b28e-cf7a9d5e2110 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:18.557093 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a3b119-9c9e-4a20-b28e-cf7a9d5e2110 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:18.557836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3b119-9c9e-4a20-b28e-cf7a9d5e2110 None admin] RBAC: Adding query filter params (name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:18.558153 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3b119-9c9e-4a20-b28e-cf7a9d5e2110 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:18.559558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a3b119-9c9e-4a20-b28e-cf7a9d5e2110 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:18.568692 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 339/668] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:48:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:18.569578 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3b119-9c9e-4a20-b28e-cf7a9d5e2110 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:18.575610 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 330/669] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:48:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:48:18.576697 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23decae3-8b00-4de2-99bf-a5fcda20a86a None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:18.609017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:18.674817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:18.681100 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a3b119-9c9e-4a20-b28e-cf7a9d5e2110 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:18.685428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23decae3-8b00-4de2-99bf-a5fcda20a86a None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:18.719034 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ad50f-127e-46a0-a1fa-cf4507821869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:18.719359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ed6e0-a970-42c7-a755-c41e9ec5b5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:18.719707 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ad50f-127e-46a0-a1fa-cf4507821869 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:18.719986 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ed6e0-a970-42c7-a755-c41e9ec5b5c6 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:18.720085 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ad50f-127e-46a0-a1fa-cf4507821869 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:18.720268 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ed6e0-a970-42c7-a755-c41e9ec5b5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:18.720363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4ad50f-127e-46a0-a1fa-cf4507821869 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:18.720543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ed6e0-a970-42c7-a755-c41e9ec5b5c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:18.721123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ad50f-127e-46a0-a1fa-cf4507821869 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:18.721392 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ed6e0-a970-42c7-a755-c41e9ec5b5c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:18.721484 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ad50f-127e-46a0-a1fa-cf4507821869 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:18.721699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ed6e0-a970-42c7-a755-c41e9ec5b5c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:18.722810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4ad50f-127e-46a0-a1fa-cf4507821869 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:18.723155 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ed6e0-a970-42c7-a755-c41e9ec5b5c6 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:18.735053 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 331/670] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:18.735699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ed6e0-a970-42c7-a755-c41e9ec5b5c6 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:18.744809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 340/671] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:18.744809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ad50f-127e-46a0-a1fa-cf4507821869 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:18.844097 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ed6e0-a970-42c7-a755-c41e9ec5b5c6 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:18.847025 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:18.853846 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4ad50f-127e-46a0-a1fa-cf4507821869 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:18.878715 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbc017-6787-4c61-b149-a72aa9c7789e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:18.879511 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbc017-6787-4c61-b149-a72aa9c7789e None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:18.879796 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbc017-6787-4c61-b149-a72aa9c7789e None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:18.880073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbc017-6787-4c61-b149-a72aa9c7789e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:18.890412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a522d-4249-4eac-bc20-5ba3602db914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:18.891238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a522d-4249-4eac-bc20-5ba3602db914 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:18.891524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a522d-4249-4eac-bc20-5ba3602db914 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:18.891804 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300a522d-4249-4eac-bc20-5ba3602db914 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:18.892026 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cbc017-6787-4c61-b149-a72aa9c7789e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:18.893924 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cbc017-6787-4c61-b149-a72aa9c7789e None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:18.902177 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a522d-4249-4eac-bc20-5ba3602db914 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:18.903918 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300a522d-4249-4eac-bc20-5ba3602db914 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:18.913841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-a5cbc017-6787-4c61-b149-a72aa9c7789e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:18.913841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:18.913841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:18.913841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:18.913841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:18.913841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:18.913841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:18.913841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:18.913841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:18.913841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:18.913841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:18.913841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:18.913841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:18.913841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:18.913841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:18.914863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:18.914863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:18.914863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:18.914863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:18.914863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:18.914863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:18.914863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:18.914863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:18.914863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:18.914863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:18.914863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:18.914863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:18.914863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:18.914863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:18.914863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:18.914863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:18.914863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:18.916037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:18.916037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:18.916037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:18.916037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:18.922073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 332/672] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:18.922195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-300a522d-4249-4eac-bc20-5ba3602db914 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:18.922195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:18.922195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:18.922195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:18.922195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:18.922195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:18.922195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:18.922195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:18.922195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:18.922195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:18.922195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:18.922195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:18.922195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:18.922195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:18.922195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:18.923214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:18.923214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:18.923214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:18.923214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:18.923214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:18.923214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:18.923214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:18.923214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:18.923214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:18.923214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:18.923214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:18.923214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:18.923214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:18.923214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:18.923214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:18.923214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:18.923214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:18.924096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:18.924096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:18.924096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:18.924096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:18.924096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 341/673] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:18.924096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a522d-4249-4eac-bc20-5ba3602db914 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:18.925249 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbc017-6787-4c61-b149-a72aa9c7789e None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:18.928420 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:19.037969 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbc017-6787-4c61-b149-a72aa9c7789e None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:19.043437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300a522d-4249-4eac-bc20-5ba3602db914 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:19.076436 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a34e8-c3d4-4660-92e4-d91fc332dc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:19.077610 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a34e8-c3d4-4660-92e4-d91fc332dc2d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:19.078211 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a34e8-c3d4-4660-92e4-d91fc332dc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:19.078498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528a34e8-c3d4-4660-92e4-d91fc332dc2d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:19.079634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1a484-0778-4aef-84db-95cb73999e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:19.080299 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1a484-0778-4aef-84db-95cb73999e5b None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:19.080595 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1a484-0778-4aef-84db-95cb73999e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:19.080876 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab1a484-0778-4aef-84db-95cb73999e5b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:19.081710 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1a484-0778-4aef-84db-95cb73999e5b None admin] RBAC: Adding query filter params (role.id=ec841a43f9924fa1b2209bc5ecea5e04, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=915f4296504b47d796e3a5b99550eb75) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:19.082022 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1a484-0778-4aef-84db-95cb73999e5b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:19.083552 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab1a484-0778-4aef-84db-95cb73999e5b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:19.085580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a34e8-c3d4-4660-92e4-d91fc332dc2d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:19.087050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528a34e8-c3d4-4660-92e4-d91fc332dc2d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:19.089494 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 342/674] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:48:18 2022] GET /identity/v3/role_assignments?user.id=915f4296504b47d796e3a5b99550eb75&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=ec841a43f9924fa1b2209bc5ecea5e04 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:19.090246 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1a484-0778-4aef-84db-95cb73999e5b None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:19.105565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-528a34e8-c3d4-4660-92e4-d91fc332dc2d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:19.105565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:19.105565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:19.105565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:19.105565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:19.105565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:19.105565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:19.105565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:19.105565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:19.105565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:19.105565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:19.105565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:19.105565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:19.105565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:19.105565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:19.106541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:19.106541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:19.106541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:19.106541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:19.106541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:19.106541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:19.106541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:19.106541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:19.106541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:19.106541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:19.106541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:19.106541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:19.106541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:19.106541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:19.106541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:19.106541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:19.106541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:19.107842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:19.107842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:19.107842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:19.107842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:19.107842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:19.107842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:19.107842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:19.107842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:19.108494 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:19.108568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 333/675] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:19.108568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a34e8-c3d4-4660-92e4-d91fc332dc2d None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:19.180786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:19.205644 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab1a484-0778-4aef-84db-95cb73999e5b None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:19.229061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528a34e8-c3d4-4660-92e4-d91fc332dc2d None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:19.252746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80596501-d5f6-41bf-88c1-54f6b5a8fab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:19.253418 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80596501-d5f6-41bf-88c1-54f6b5a8fab0 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:19.253876 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80596501-d5f6-41bf-88c1-54f6b5a8fab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:19.254080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80596501-d5f6-41bf-88c1-54f6b5a8fab0 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:19.254944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80596501-d5f6-41bf-88c1-54f6b5a8fab0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:19.254944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80596501-d5f6-41bf-88c1-54f6b5a8fab0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:19.256659 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80596501-d5f6-41bf-88c1-54f6b5a8fab0 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:19.268030 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4711ae2-67ab-46f0-9a42-e7cbbe41e77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:19.268908 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4711ae2-67ab-46f0-9a42-e7cbbe41e77d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:19.269039 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4711ae2-67ab-46f0-9a42-e7cbbe41e77d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:19.269257 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4711ae2-67ab-46f0-9a42-e7cbbe41e77d None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:19.269988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4711ae2-67ab-46f0-9a42-e7cbbe41e77d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:19.270299 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4711ae2-67ab-46f0-9a42-e7cbbe41e77d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:19.272274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4711ae2-67ab-46f0-9a42-e7cbbe41e77d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:19.272428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 343/676] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:19.273796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595926f-894e-463c-8617-90573c3acf53 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:19.274097 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595926f-894e-463c-8617-90573c3acf53 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:19.274382 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595926f-894e-463c-8617-90573c3acf53 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:19.275901 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 344/677] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:19.276431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595926f-894e-463c-8617-90573c3acf53 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:19.289770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 334/678] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:19.290901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4711ae2-67ab-46f0-9a42-e7cbbe41e77d None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:19.354441 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:19.390487 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595926f-894e-463c-8617-90573c3acf53 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:19.407600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4711ae2-67ab-46f0-9a42-e7cbbe41e77d None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:19.425671 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7063c320-3d5c-46d3-a5a5-d2523f5275db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:19.426416 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063c320-3d5c-46d3-a5a5-d2523f5275db None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:19.426799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063c320-3d5c-46d3-a5a5-d2523f5275db None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:19.427656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7063c320-3d5c-46d3-a5a5-d2523f5275db None admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:19.428518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063c320-3d5c-46d3-a5a5-d2523f5275db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:19.428905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063c320-3d5c-46d3-a5a5-d2523f5275db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:19.430383 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7063c320-3d5c-46d3-a5a5-d2523f5275db None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:19.431766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:19.447826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 345/679] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:19.447826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a7931-26c9-4bda-a5bb-2e8cf79aaa0d None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:19.447826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a7931-26c9-4bda-a5bb-2e8cf79aaa0d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:19.447826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12a7931-26c9-4bda-a5bb-2e8cf79aaa0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:19.450483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ba6dd-b1ea-4591-be7e-3298cf15d8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:19.451312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ba6dd-b1ea-4591-be7e-3298cf15d8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:19.451607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ba6dd-b1ea-4591-be7e-3298cf15d8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:19.451890 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74ba6dd-b1ea-4591-be7e-3298cf15d8a6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:19.460878 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ba6dd-b1ea-4591-be7e-3298cf15d8a6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:19.462260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74ba6dd-b1ea-4591-be7e-3298cf15d8a6 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:19.464871 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a12a7931-26c9-4bda-a5bb-2e8cf79aaa0d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:19.466128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a12a7931-26c9-4bda-a5bb-2e8cf79aaa0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:19.476686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-c74ba6dd-b1ea-4591-be7e-3298cf15d8a6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 22:48:19.476686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:19.476686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:19.476686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:19.476686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:19.476686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:19.476686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:19.476686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:19.476686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:19.476686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:19.476686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:19.476686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:19.476686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:19.476686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:19.476686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:19.476686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:19.477664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:19.477664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:19.477664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:19.477664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:19.477664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:19.477664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:19.477664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:19.477664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:19.477664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:19.477664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:19.477664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:19.477664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:19.477664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:19.477664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:19.477664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:19.477664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:19.477664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:19.477664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:19.478654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:19.478654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:19.478654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:19.478654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:19.478654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:19.478654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:19.478654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:19.478654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:19.478654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 22:48:19.478654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:19.478654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 335/680] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:48:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:19.479271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ba6dd-b1ea-4591-be7e-3298cf15d8a6 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:19.550614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 346/681] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:19.550614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a7931-26c9-4bda-a5bb-2e8cf79aaa0d None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:19.600302 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74ba6dd-b1ea-4591-be7e-3298cf15d8a6 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:19.604344 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:19.649102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a5979-c364-48f0-9153-83c7c627d892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:19.650468 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a5979-c364-48f0-9153-83c7c627d892 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:19.650984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a5979-c364-48f0-9153-83c7c627d892 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:19.651466 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12a7931-26c9-4bda-a5bb-2e8cf79aaa0d None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:19.651693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10a5979-c364-48f0-9153-83c7c627d892 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:19.660070 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a5979-c364-48f0-9153-83c7c627d892 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:19.661533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10a5979-c364-48f0-9153-83c7c627d892 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:19.675428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-e10a5979-c364-48f0-9153-83c7c627d892 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:19.675428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:19.675428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:19.675428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:19.675428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:19.675428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:19.675428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:19.675428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:19.675428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:19.675428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:19.675428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:19.675428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:19.675428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:19.675428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:19.675428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:19.676364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:19.676364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:19.676364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:19.676364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:19.676364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:19.676364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:19.676364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:19.676364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:19.676364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:19.676364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:19.676364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:19.676364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:19.676364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:19.676364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:19.676364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:19.676364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:19.676364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:19.677400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:19.677400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:19.677400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:19.677400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:19.677400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:19.677400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:19.677400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:19.677400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:19.677400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 336/682] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:19.677400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a5979-c364-48f0-9153-83c7c627d892 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:19.692324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b98072-a48e-4d02-b305-3e9489ccd5a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:19.693794 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b98072-a48e-4d02-b305-3e9489ccd5a7 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:19.693794 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b98072-a48e-4d02-b305-3e9489ccd5a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:19.694094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b98072-a48e-4d02-b305-3e9489ccd5a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:19.694937 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b98072-a48e-4d02-b305-3e9489ccd5a7 None admin] RBAC: Adding query filter params (role.id=ec841a43f9924fa1b2209bc5ecea5e04, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=24ca7f221334442781f10574277bf422) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:19.695254 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b98072-a48e-4d02-b305-3e9489ccd5a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:19.696688 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b98072-a48e-4d02-b305-3e9489ccd5a7 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:19.710629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 347/683] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:48:19 2022] GET /identity/v3/role_assignments?user.id=24ca7f221334442781f10574277bf422&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=ec841a43f9924fa1b2209bc5ecea5e04 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:19.710629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b98072-a48e-4d02-b305-3e9489ccd5a7 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:19.733221 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:19.815632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10a5979-c364-48f0-9153-83c7c627d892 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:19.836527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b98072-a48e-4d02-b305-3e9489ccd5a7 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:19.854271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc376bd6-088d-4acf-b403-dde41d1501e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:19.855257 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc376bd6-088d-4acf-b403-dde41d1501e0 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:19.855613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc376bd6-088d-4acf-b403-dde41d1501e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:19.856349 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc376bd6-088d-4acf-b403-dde41d1501e0 None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:19.857605 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc376bd6-088d-4acf-b403-dde41d1501e0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:19.858037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc376bd6-088d-4acf-b403-dde41d1501e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:19.860163 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc376bd6-088d-4acf-b403-dde41d1501e0 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:19.870495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:19.884129 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f2203-f24e-4171-9667-35b3a925878a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:19.885028 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f2203-f24e-4171-9667-35b3a925878a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:19.885381 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f2203-f24e-4171-9667-35b3a925878a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:19.885660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 337/684] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jun 16 22:48:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:19.886779 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc376bd6-088d-4acf-b403-dde41d1501e0 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:19.886888 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22f2203-f24e-4171-9667-35b3a925878a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:19.899599 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22f2203-f24e-4171-9667-35b3a925878a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:19.910444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-a22f2203-f24e-4171-9667-35b3a925878a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:48:19.910444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:19.910444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:19.910444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:19.910444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:19.910444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:19.910444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:19.910444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:19.910444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:19.910444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:19.910444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:19.910444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:19.910444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:19.910444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:19.910444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:19.910444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:19.911623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:19.911623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:19.911623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:19.911623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:19.911623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:19.911623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:19.911623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:19.911623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:19.911623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:19.911623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:19.911623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:19.911623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:19.911623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:19.911623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:19.911623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:19.911623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:19.912570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:19.912570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:19.912570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:19.912570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:19.912570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:19.912570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:19.912570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:19.912570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:19.912570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:19.912570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:19.912570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:48:19.912570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:19.914670 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 348/685] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:48:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:19.915622 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f2203-f24e-4171-9667-35b3a925878a None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:20.009597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:20.014000 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc376bd6-088d-4acf-b403-dde41d1501e0 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:20.042947 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22f2203-f24e-4171-9667-35b3a925878a None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:20.055072 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8dc036-e5d9-478c-a752-a976743363fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:20.056299 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8dc036-e5d9-478c-a752-a976743363fc None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:20.056684 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8dc036-e5d9-478c-a752-a976743363fc None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:20.057342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8dc036-e5d9-478c-a752-a976743363fc None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:20.058236 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8dc036-e5d9-478c-a752-a976743363fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:20.058669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8dc036-e5d9-478c-a752-a976743363fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:20.061277 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8dc036-e5d9-478c-a752-a976743363fc None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:20.077780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 338/686] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:20.079095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8dc036-e5d9-478c-a752-a976743363fc None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:20.083070 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5ff39-a850-4ea6-9110-97edd5f347eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:20.084547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5ff39-a850-4ea6-9110-97edd5f347eb None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:20.085016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5ff39-a850-4ea6-9110-97edd5f347eb None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:20.085474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff5ff39-a850-4ea6-9110-97edd5f347eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:20.148365 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:20.164632 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5ff39-a850-4ea6-9110-97edd5f347eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:20.166579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff5ff39-a850-4ea6-9110-97edd5f347eb None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:20.206082 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8dc036-e5d9-478c-a752-a976743363fc None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:20.247944 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eff5dd-db33-4f5a-9a44-9e0d0e52854a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:20.248625 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eff5dd-db33-4f5a-9a44-9e0d0e52854a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:20.248910 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eff5dd-db33-4f5a-9a44-9e0d0e52854a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:20.249186 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2eff5dd-db33-4f5a-9a44-9e0d0e52854a None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:20.249930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eff5dd-db33-4f5a-9a44-9e0d0e52854a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:20.250238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eff5dd-db33-4f5a-9a44-9e0d0e52854a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:20.252120 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2eff5dd-db33-4f5a-9a44-9e0d0e52854a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:20.266668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 339/687] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:48:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:20.267678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eff5dd-db33-4f5a-9a44-9e0d0e52854a None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:20.283104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 349/688] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:48:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:48:20.284354 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5ff39-a850-4ea6-9110-97edd5f347eb None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:20.298620 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:20.375371 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:20.377354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2eff5dd-db33-4f5a-9a44-9e0d0e52854a None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:20.393072 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff5ff39-a850-4ea6-9110-97edd5f347eb None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:20.415349 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103779f6-f69b-4e5c-be27-41adf4803925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:20.416309 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103779f6-f69b-4e5c-be27-41adf4803925 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:20.416790 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103779f6-f69b-4e5c-be27-41adf4803925 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:20.417281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103779f6-f69b-4e5c-be27-41adf4803925 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:20.418397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103779f6-f69b-4e5c-be27-41adf4803925 None admin] RBAC: Adding query filter params (role.id=ec841a43f9924fa1b2209bc5ecea5e04, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=6b794f7aa2a7488e91f7d9ea35a45d78) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:20.418997 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103779f6-f69b-4e5c-be27-41adf4803925 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:20.420629 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103779f6-f69b-4e5c-be27-41adf4803925 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:20.427441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 340/689] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:48:20 2022] GET /identity/v3/role_assignments?user.id=6b794f7aa2a7488e91f7d9ea35a45d78&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=ec841a43f9924fa1b2209bc5ecea5e04 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:20.428537 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103779f6-f69b-4e5c-be27-41adf4803925 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:20.429849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3b03a-b77e-4341-b541-069ab89a3122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:20.430690 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3b03a-b77e-4341-b541-069ab89a3122 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:20.430984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3b03a-b77e-4341-b541-069ab89a3122 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:20.431268 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e3b03a-b77e-4341-b541-069ab89a3122 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:20.443794 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3b03a-b77e-4341-b541-069ab89a3122 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:20.445303 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e3b03a-b77e-4341-b541-069ab89a3122 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:20.464482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-50e3b03a-b77e-4341-b541-069ab89a3122 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:20.464482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:20.464482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:20.464482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:20.464482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:20.464482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:20.464482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:20.464482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:20.464482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:20.464482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:20.464482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:20.464482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:20.464482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:20.464482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:20.464482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:20.465496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:20.465496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:20.465496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:20.465496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:20.465496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:20.465496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:20.465496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:20.465496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:20.465496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:20.465496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:20.465496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:20.465496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:20.465496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:20.465496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:20.465496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:20.465496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:20.465496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:20.466360 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:20.466360 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:20.466360 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:20.466360 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:20.466360 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 350/690] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:20.472142 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3b03a-b77e-4341-b541-069ab89a3122 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:20.531053 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:20.545789 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103779f6-f69b-4e5c-be27-41adf4803925 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:20.585236 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d52b10-b5b0-4ad9-99ce-e83d373af30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:20.586466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d52b10-b5b0-4ad9-99ce-e83d373af30e None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:20.587027 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d52b10-b5b0-4ad9-99ce-e83d373af30e None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:20.587496 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d52b10-b5b0-4ad9-99ce-e83d373af30e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:20.614330 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d52b10-b5b0-4ad9-99ce-e83d373af30e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:20.616431 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d52b10-b5b0-4ad9-99ce-e83d373af30e None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:20.644063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-59d52b10-b5b0-4ad9-99ce-e83d373af30e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:20.644063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:20.644063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:20.644063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:20.644063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:20.644063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:20.644063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:20.644063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:20.644063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:20.644063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:20.644063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:20.644063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:20.644063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:20.644063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:20.644063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:20.644978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:20.644978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:20.644978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:20.644978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:20.644978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:20.644978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:20.644978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:20.644978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:20.644978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:20.644978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:20.644978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:20.644978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:20.644978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:20.644978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:20.644978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:20.644978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:20.644978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:20.645888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:20.645888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:20.645888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:20.645888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:20.648347 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 341/691] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:20.651512 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e3b03a-b77e-4341-b541-069ab89a3122 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:20.654773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d52b10-b5b0-4ad9-99ce-e83d373af30e None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:20.695024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e44ca2-2c4f-4bb4-a666-32b0edd62f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:20.695622 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e44ca2-2c4f-4bb4-a666-32b0edd62f82 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:20.695958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e44ca2-2c4f-4bb4-a666-32b0edd62f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:20.696174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e44ca2-2c4f-4bb4-a666-32b0edd62f82 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:20.696898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e44ca2-2c4f-4bb4-a666-32b0edd62f82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:20.697207 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e44ca2-2c4f-4bb4-a666-32b0edd62f82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:20.698583 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e44ca2-2c4f-4bb4-a666-32b0edd62f82 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:20.706693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:20.713392 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 351/692] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:20.720572 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e44ca2-2c4f-4bb4-a666-32b0edd62f82 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:20.770070 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d52b10-b5b0-4ad9-99ce-e83d373af30e None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:20.809018 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03da86f-d49b-41d9-baee-e7de0effbe3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:20.810366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03da86f-d49b-41d9-baee-e7de0effbe3a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:20.810915 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03da86f-d49b-41d9-baee-e7de0effbe3a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:20.811652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03da86f-d49b-41d9-baee-e7de0effbe3a None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:20.812961 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03da86f-d49b-41d9-baee-e7de0effbe3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:20.813471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03da86f-d49b-41d9-baee-e7de0effbe3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:20.815574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03da86f-d49b-41d9-baee-e7de0effbe3a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:20.830836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e44ca2-2c4f-4bb4-a666-32b0edd62f82 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:20.832095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 342/693] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:20.836722 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:20.837812 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03da86f-d49b-41d9-baee-e7de0effbe3a None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:20.876101 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3d617-11fe-4ade-bdf1-b4fdd67c86bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:20.876905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3d617-11fe-4ade-bdf1-b4fdd67c86bf None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:20.877189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3d617-11fe-4ade-bdf1-b4fdd67c86bf None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:20.877463 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e3d617-11fe-4ade-bdf1-b4fdd67c86bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:20.891801 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3d617-11fe-4ade-bdf1-b4fdd67c86bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:20.892843 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e3d617-11fe-4ade-bdf1-b4fdd67c86bf None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:20.926791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-96e3d617-11fe-4ade-bdf1-b4fdd67c86bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 22:48:20.926791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:20.926791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:20.926791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:20.926791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:20.926791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:20.926791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:20.926791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:20.926791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:20.926791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:20.926791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:20.926791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:20.926791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:20.926791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:20.926791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:20.926791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:20.928346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:20.928346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:20.928346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:20.928346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:20.928346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:20.928346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:20.928346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:20.928346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:20.928346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:20.928346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:20.928346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:20.928346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:20.928346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:20.928346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:20.928346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:20.928346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:20.928346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:20.928346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:20.929272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:20.929272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:20.929272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:20.929272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:20.929272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:20.929272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:20.929272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:20.929272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:20.929272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 22:48:20.929272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:20.939039 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 352/694] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jun 16 22:48:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:20.944501 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3d617-11fe-4ade-bdf1-b4fdd67c86bf None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:20.965260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03da86f-d49b-41d9-baee-e7de0effbe3a None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:21.003522 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af595ed-e38c-4132-a92e-bbaeb25f2125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:21.004745 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af595ed-e38c-4132-a92e-bbaeb25f2125 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:21.005282 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af595ed-e38c-4132-a92e-bbaeb25f2125 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:21.005760 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af595ed-e38c-4132-a92e-bbaeb25f2125 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:21.014174 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af595ed-e38c-4132-a92e-bbaeb25f2125 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:21.016586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af595ed-e38c-4132-a92e-bbaeb25f2125 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:21.032946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-0af595ed-e38c-4132-a92e-bbaeb25f2125 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:21.032946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:21.032946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:21.032946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:21.032946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:21.032946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:21.032946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:21.032946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:21.032946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:21.032946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:21.032946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:21.032946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:21.032946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:21.032946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:21.032946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:21.033866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:21.033866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:21.033866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:21.033866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:21.033866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:21.033866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:21.033866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:21.033866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:21.033866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:21.033866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:21.033866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:21.033866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:21.033866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:21.033866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:21.033866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:21.033866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:21.033866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:21.034786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:21.034786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:21.034786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:21.034786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:21.034786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:21.034786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:21.034786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:21.034786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:21.037740 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 343/695] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:21.043514 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af595ed-e38c-4132-a92e-bbaeb25f2125 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:21.054846 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:21.099491 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e3d617-11fe-4ade-bdf1-b4fdd67c86bf None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:21.111010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:21.150492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af595ed-e38c-4132-a92e-bbaeb25f2125 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:21.155272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d45b73-6ed2-45a6-b03d-7618d4ccd99f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:21.155965 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d45b73-6ed2-45a6-b03d-7618d4ccd99f None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:21.156252 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d45b73-6ed2-45a6-b03d-7618d4ccd99f None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:21.156526 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d45b73-6ed2-45a6-b03d-7618d4ccd99f None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:21.157367 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d45b73-6ed2-45a6-b03d-7618d4ccd99f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:21.157675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d45b73-6ed2-45a6-b03d-7618d4ccd99f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:21.159135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d45b73-6ed2-45a6-b03d-7618d4ccd99f None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:21.187761 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87262d54-9fb1-4e57-b0ac-6c1165c94057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:21.188836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87262d54-9fb1-4e57-b0ac-6c1165c94057 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:21.189335 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87262d54-9fb1-4e57-b0ac-6c1165c94057 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:21.189820 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87262d54-9fb1-4e57-b0ac-6c1165c94057 None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:21.190835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87262d54-9fb1-4e57-b0ac-6c1165c94057 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:21.191365 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87262d54-9fb1-4e57-b0ac-6c1165c94057 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:21.193597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87262d54-9fb1-4e57-b0ac-6c1165c94057 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:21.195575 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 353/696] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jun 16 22:48:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:21.199760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d45b73-6ed2-45a6-b03d-7618d4ccd99f None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:21.207412 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 344/697] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:21.213385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87262d54-9fb1-4e57-b0ac-6c1165c94057 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:21.317360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87262d54-9fb1-4e57-b0ac-6c1165c94057 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:21.343396 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d45b73-6ed2-45a6-b03d-7618d4ccd99f None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:21.354997 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:21.358655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23012c63-fd58-4d92-a3f4-4fb2f20b9efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:21.359583 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23012c63-fd58-4d92-a3f4-4fb2f20b9efc None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:21.360083 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23012c63-fd58-4d92-a3f4-4fb2f20b9efc None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:21.360555 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23012c63-fd58-4d92-a3f4-4fb2f20b9efc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:21.361295 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:21.361817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23012c63-fd58-4d92-a3f4-4fb2f20b9efc None admin] RBAC: Adding query filter params (role.id=ec841a43f9924fa1b2209bc5ecea5e04, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=bef518c9430e4274b1066241f821f4a1) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:21.362558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23012c63-fd58-4d92-a3f4-4fb2f20b9efc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:21.364550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23012c63-fd58-4d92-a3f4-4fb2f20b9efc None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:21.371915 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 345/698] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:48:21 2022] GET /identity/v3/role_assignments?user.id=bef518c9430e4274b1066241f821f4a1&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=ec841a43f9924fa1b2209bc5ecea5e04 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:21.381945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ae486-e757-482e-a265-3344fb865fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:21.382689 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ae486-e757-482e-a265-3344fb865fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:21.382987 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ae486-e757-482e-a265-3344fb865fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:21.383268 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877ae486-e757-482e-a265-3344fb865fa2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:21.386431 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e23703-5e05-4fa1-ae64-a0eafee7a958 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:21.386833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e23703-5e05-4fa1-ae64-a0eafee7a958 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:21.387083 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e23703-5e05-4fa1-ae64-a0eafee7a958 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:21.388736 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 346/699] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:48:21.394248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93942c-5922-4de2-9f8b-b016a0da49cd None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:21.394536 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93942c-5922-4de2-9f8b-b016a0da49cd None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:21.394854 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93942c-5922-4de2-9f8b-b016a0da49cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:21.413187 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-0c93942c-5922-4de2-9f8b-b016a0da49cd None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:21.414404 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-0c93942c-5922-4de2-9f8b-b016a0da49cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:21.460550 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ae486-e757-482e-a265-3344fb865fa2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:21.462026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-877ae486-e757-482e-a265-3344fb865fa2 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:21.578345 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 347/700] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:48:21.590874 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 354/701] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:48:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:48:21.596518 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93942c-5922-4de2-9f8b-b016a0da49cd None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:21.611176 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ae486-e757-482e-a265-3344fb865fa2 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:21.632431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:21.727689 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877ae486-e757-482e-a265-3344fb865fa2 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:21.734610 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93942c-5922-4de2-9f8b-b016a0da49cd None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:21.752746 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:21.770492 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981997c2-1c8e-4d7e-a407-a095c926a643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:21.771259 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981997c2-1c8e-4d7e-a407-a095c926a643 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:21.771551 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981997c2-1c8e-4d7e-a407-a095c926a643 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:21.771836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981997c2-1c8e-4d7e-a407-a095c926a643 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:21.780287 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a617eca-637b-485d-b677-f5094357981f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:21.781361 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a617eca-637b-485d-b677-f5094357981f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:21.782255 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a617eca-637b-485d-b677-f5094357981f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:21.782919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a617eca-637b-485d-b677-f5094357981f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:21.783900 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981997c2-1c8e-4d7e-a407-a095c926a643 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:21.785773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981997c2-1c8e-4d7e-a407-a095c926a643 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:21.795887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a617eca-637b-485d-b677-f5094357981f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:21.798239 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a617eca-637b-485d-b677-f5094357981f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:21.806984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-981997c2-1c8e-4d7e-a407-a095c926a643 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:21.806984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:21.806984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:21.806984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:21.806984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:21.806984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:21.806984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:21.806984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:21.806984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:21.806984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:21.806984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:21.806984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:21.806984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:21.806984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:21.806984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:21.807911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:21.807911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:21.807911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:21.807911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:21.807911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:21.807911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:21.807911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:21.807911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:21.807911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:21.807911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:21.807911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:21.807911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:21.807911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:21.807911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:21.807911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:21.807911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:21.807911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:21.808881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:21.808881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:21.808881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:21.808881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:21.808881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 355/702] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:21.814147 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981997c2-1c8e-4d7e-a407-a095c926a643 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:21.821208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-8a617eca-637b-485d-b677-f5094357981f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:21.821208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:21.821208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:21.821208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:21.821208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:21.821208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:21.821208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:21.821208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:21.821208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:21.821208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:21.821208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:21.821208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:21.821208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:21.821208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:21.821208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:21.822073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:21.822073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:21.822073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:21.822073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:21.822073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:21.822073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:21.822073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:21.822073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:21.822073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:21.822073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:21.822073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:21.822073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:21.822073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:21.822073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:21.822073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:21.822073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:21.822073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:21.823102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:21.823102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:21.823102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:21.823102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:21.825400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 348/703] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:21.831671 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a617eca-637b-485d-b677-f5094357981f None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:21.904711 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:21.948605 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981997c2-1c8e-4d7e-a407-a095c926a643 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:21.951994 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a617eca-637b-485d-b677-f5094357981f None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:21.985777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f32b5-0a9e-414e-bf6f-1fdfff6978a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:21.986785 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f32b5-0a9e-414e-bf6f-1fdfff6978a0 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:21.987255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f32b5-0a9e-414e-bf6f-1fdfff6978a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:21.987784 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59f32b5-0a9e-414e-bf6f-1fdfff6978a0 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:21.988713 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59f32b5-0a9e-414e-bf6f-1fdfff6978a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:21.989226 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59f32b5-0a9e-414e-bf6f-1fdfff6978a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:21.990851 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59f32b5-0a9e-414e-bf6f-1fdfff6978a0 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:21.991567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1021166-a649-4040-85ad-dea0c18aa48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:21.992254 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1021166-a649-4040-85ad-dea0c18aa48a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:21.992549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1021166-a649-4040-85ad-dea0c18aa48a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:21.992833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1021166-a649-4040-85ad-dea0c18aa48a None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:21.993669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1021166-a649-4040-85ad-dea0c18aa48a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:21.993992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1021166-a649-4040-85ad-dea0c18aa48a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:21.995541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1021166-a649-4040-85ad-dea0c18aa48a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:22.005063 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 356/704] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:22.011507 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f32b5-0a9e-414e-bf6f-1fdfff6978a0 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:22.013519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 349/705] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:22.019140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1021166-a649-4040-85ad-dea0c18aa48a None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:22.048981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:22.138051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1021166-a649-4040-85ad-dea0c18aa48a None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:22.140638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59f32b5-0a9e-414e-bf6f-1fdfff6978a0 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:22.176695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50c191-052d-402b-bbb9-ae3ceef54cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:22.178462 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50c191-052d-402b-bbb9-ae3ceef54cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:22.178962 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50c191-052d-402b-bbb9-ae3ceef54cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:22.179449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d50c191-052d-402b-bbb9-ae3ceef54cd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:22.180082 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:22.189772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ada2063-b013-41eb-ba39-4a1b29ee320b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:22.190544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada2063-b013-41eb-ba39-4a1b29ee320b None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:22.190875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50c191-052d-402b-bbb9-ae3ceef54cd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:22.191022 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada2063-b013-41eb-ba39-4a1b29ee320b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:22.191331 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ada2063-b013-41eb-ba39-4a1b29ee320b None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:22.192103 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada2063-b013-41eb-ba39-4a1b29ee320b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:22.193019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d50c191-052d-402b-bbb9-ae3ceef54cd2 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:22.193704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ada2063-b013-41eb-ba39-4a1b29ee320b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:22.209942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-0d50c191-052d-402b-bbb9-ae3ceef54cd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:22.209942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:22.209942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:22.209942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:22.209942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:22.209942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:22.209942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:22.209942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:22.209942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:22.209942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:22.209942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:22.209942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:22.209942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:22.209942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:22.209942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:22.211021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:22.211021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:22.211021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:22.211021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:22.211021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:22.211021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:22.211021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:22.211021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:22.211021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:22.211021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:22.211021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:22.211021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:22.211021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:22.211021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:22.211021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:22.211021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:22.211021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:22.211921 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:22.211921 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:22.211921 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:22.211921 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:22.211921 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:22.211921 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:22.211921 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:22.211921 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:22.214027 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 350/706] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:22.219792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50c191-052d-402b-bbb9-ae3ceef54cd2 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:22.237842 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 357/707] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 22:48:22 2022] POST /identity/v3/users => generated 254 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:48:22.329616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b0f43-f7c3-4cf8-ac9b-59cbcd6ae96c None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:22.329910 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b0f43-f7c3-4cf8-ac9b-59cbcd6ae96c None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:22.330186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9b0f43-f7c3-4cf8-ac9b-59cbcd6ae96c None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:22.334727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 358/708] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:48:22.337768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ceb59f-a984-4052-8638-e86bbbb9a7d9 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:22.338062 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ceb59f-a984-4052-8638-e86bbbb9a7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:22.338342 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ceb59f-a984-4052-8638-e86bbbb9a7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:22.360589 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:22.380956 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d50c191-052d-402b-bbb9-ae3ceef54cd2 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:22.383651 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-45ceb59f-a984-4052-8638-e86bbbb9a7d9 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:22.385266 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-45ceb59f-a984-4052-8638-e86bbbb9a7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:22.422313 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b739b6-2398-4a77-988f-90024db12b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:22.423300 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b739b6-2398-4a77-988f-90024db12b26 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:22.423737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b739b6-2398-4a77-988f-90024db12b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:22.424180 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b739b6-2398-4a77-988f-90024db12b26 None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:22.425313 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b739b6-2398-4a77-988f-90024db12b26 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:22.425779 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b739b6-2398-4a77-988f-90024db12b26 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:22.428108 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82b739b6-2398-4a77-988f-90024db12b26 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:22.442441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 351/709] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:22.448745 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b739b6-2398-4a77-988f-90024db12b26 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:22.524258 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 359/710] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:48:22.554277 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ceb59f-a984-4052-8638-e86bbbb9a7d9 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:22.583332 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b739b6-2398-4a77-988f-90024db12b26 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:22.625684 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fb490-1495-417f-bcd7-d7fbd9ae0326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:22.626743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fb490-1495-417f-bcd7-d7fbd9ae0326 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:22.627288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fb490-1495-417f-bcd7-d7fbd9ae0326 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:22.627773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8fb490-1495-417f-bcd7-d7fbd9ae0326 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:22.628831 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fb490-1495-417f-bcd7-d7fbd9ae0326 None admin] RBAC: Adding query filter params (role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=4180f2593149437fa27effd8d1338d10) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:22.629361 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fb490-1495-417f-bcd7-d7fbd9ae0326 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:22.632407 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8fb490-1495-417f-bcd7-d7fbd9ae0326 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:22.639363 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 352/711] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:48:22 2022] GET /identity/v3/role_assignments?user.id=4180f2593149437fa27effd8d1338d10&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:22.682297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ceb59f-a984-4052-8638-e86bbbb9a7d9 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:22.699112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:22.723302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385659a-f2e7-49fb-89ab-617f4b5c7aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:22.724175 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385659a-f2e7-49fb-89ab-617f4b5c7aaf None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:22.724460 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385659a-f2e7-49fb-89ab-617f4b5c7aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:22.724736 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1385659a-f2e7-49fb-89ab-617f4b5c7aaf None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:22.725481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385659a-f2e7-49fb-89ab-617f4b5c7aaf None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:22.726983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1385659a-f2e7-49fb-89ab-617f4b5c7aaf None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:22.742400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-1385659a-f2e7-49fb-89ab-617f4b5c7aaf None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 22:48:22.742400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:22.742400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:22.742400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:22.742400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:22.742400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:22.742400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:22.742400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:22.742400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 22:48:22.742400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:22.742400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:22.742400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:22.742400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:22.742400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:22.742400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:22.742400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:22.743458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:22.743458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:22.743458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:22.743458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:22.743458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:22.743458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:22.743458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:22.743458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:22.743458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:22.743458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:22.743458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:22.743458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:22.743458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:22.743458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:22.743458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:22.743458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:22.744302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:22.744302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:22.744302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:22.744302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:22.744302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:22.744302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:22.744302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:22.744302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:22.744302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:22.744302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:22.744302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:22.744302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:22.744302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:22.744302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 22:48:22.744302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:22.744302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 360/712] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jun 16 22:48:22 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:22.751585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fb490-1495-417f-bcd7-d7fbd9ae0326 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:22.778926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:22.896785 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8fb490-1495-417f-bcd7-d7fbd9ae0326 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:22.932555 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a0e17-2942-49f0-a8a6-385ddd247faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:22.933148 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a0e17-2942-49f0-a8a6-385ddd247faf None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:22.933431 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a0e17-2942-49f0-a8a6-385ddd247faf None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:22.933709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a0e17-2942-49f0-a8a6-385ddd247faf None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:22.934415 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a0e17-2942-49f0-a8a6-385ddd247faf None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:22.934749 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a0e17-2942-49f0-a8a6-385ddd247faf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:22.936117 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446a0e17-2942-49f0-a8a6-385ddd247faf None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:22.941147 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 353/713] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jun 16 22:48:22 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:22.947249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385659a-f2e7-49fb-89ab-617f4b5c7aaf None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:23.070487 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1385659a-f2e7-49fb-89ab-617f4b5c7aaf None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:23.120404 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f8505-41b5-48c0-ba9d-27d64f20b0a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:23.121599 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f8505-41b5-48c0-ba9d-27d64f20b0a6 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:23.122177 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f8505-41b5-48c0-ba9d-27d64f20b0a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:23.122725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f8505-41b5-48c0-ba9d-27d64f20b0a6 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:23.123705 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f8505-41b5-48c0-ba9d-27d64f20b0a6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:23.124216 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f8505-41b5-48c0-ba9d-27d64f20b0a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:23.125849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969f8505-41b5-48c0-ba9d-27d64f20b0a6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:23.135674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 361/714] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jun 16 22:48:22 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:23.338099 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d68f1-5df6-43bb-a184-21d8c97f801b None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:23.339093 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d68f1-5df6-43bb-a184-21d8c97f801b None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:23.339577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d68f1-5df6-43bb-a184-21d8c97f801b None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:23.341396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 354/715] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:23.347098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0eda5-a159-45e3-a8d5-7bbd4e576056 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:23.347388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0eda5-a159-45e3-a8d5-7bbd4e576056 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:23.347669 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0eda5-a159-45e3-a8d5-7bbd4e576056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:23.372395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-1ee0eda5-a159-45e3-a8d5-7bbd4e576056 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:23.373609 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-1ee0eda5-a159-45e3-a8d5-7bbd4e576056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:23.422213 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:23.459978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 362/716] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:23.483683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d68f1-5df6-43bb-a184-21d8c97f801b None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:23.577693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:23.619493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8d68f1-5df6-43bb-a184-21d8c97f801b None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:23.667913 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11fe97-bd88-490a-a1ea-9c7a0b7e886f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:23.668775 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11fe97-bd88-490a-a1ea-9c7a0b7e886f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:23.669119 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11fe97-bd88-490a-a1ea-9c7a0b7e886f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:23.669405 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11fe97-bd88-490a-a1ea-9c7a0b7e886f None admin] PATH_INFO: `/v3/services/network` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:23.670060 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11fe97-bd88-490a-a1ea-9c7a0b7e886f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:23.671456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f11fe97-bd88-490a-a1ea-9c7a0b7e886f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:23.690952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-4f11fe97-bd88-490a-a1ea-9c7a0b7e886f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 22:48:23.690952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:23.690952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:23.690952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:23.690952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:23.690952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:23.690952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:23.690952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:23.690952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 22:48:23.690952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:23.690952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:23.690952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:23.690952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:23.690952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:23.690952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:23.690952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:23.690952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:23.692096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:23.692096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:23.692096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:23.692096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:23.692096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:23.692096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:23.692096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:23.692096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:23.692096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:23.692096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:23.692096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:23.692096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:23.692096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:23.692096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:23.692096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:23.692096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:23.693038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:23.693038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:23.693038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:23.693038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:23.693038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:23.693038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:23.693038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:23.693038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:23.693038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:23.693038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:23.693038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:23.693038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:23.693038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 22:48:23.693038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:23.693038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 355/717] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:23 2022] GET /identity/v3/services/network => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:23.703687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0eda5-a159-45e3-a8d5-7bbd4e576056 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:23.858841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0eda5-a159-45e3-a8d5-7bbd4e576056 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:23.908637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cb9ac8-3442-456c-8436-c4e3932a832d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:23.909313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb9ac8-3442-456c-8436-c4e3932a832d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:23.909600 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb9ac8-3442-456c-8436-c4e3932a832d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:23.909878 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cb9ac8-3442-456c-8436-c4e3932a832d None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:23.910581 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb9ac8-3442-456c-8436-c4e3932a832d None admin] RBAC: Adding query filter params (name=network) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:23.910908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb9ac8-3442-456c-8436-c4e3932a832d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:23.912449 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cb9ac8-3442-456c-8436-c4e3932a832d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:23.918352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 363/718] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 22:48:23 2022] GET /identity/v3/services?name=network => generated 124 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:23.922933 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11fe97-bd88-490a-a1ea-9c7a0b7e886f None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:24.053555 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11fe97-bd88-490a-a1ea-9c7a0b7e886f None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:24.089936 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54affb7-8bfe-4c93-b2a9-d7baba369a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:24.091039 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54affb7-8bfe-4c93-b2a9-d7baba369a9e None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:24.091602 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54affb7-8bfe-4c93-b2a9-d7baba369a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:24.092099 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54affb7-8bfe-4c93-b2a9-d7baba369a9e None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:24.093011 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54affb7-8bfe-4c93-b2a9-d7baba369a9e None admin] RBAC: Adding query filter params (type=network) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:24.093521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54affb7-8bfe-4c93-b2a9-d7baba369a9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:24.095144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54affb7-8bfe-4c93-b2a9-d7baba369a9e None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:24.098954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:24.102249 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 356/719] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 22:48:23 2022] GET /identity/v3/services?type=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:24.385238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d71f8f-f538-4819-a3f2-f5c3454f7bee None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:24.385763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d71f8f-f538-4819-a3f2-f5c3454f7bee None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:24.385763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d71f8f-f538-4819-a3f2-f5c3454f7bee None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:24.387456 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 364/720] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:24.392373 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024cdb5-f253-400f-a2b9-3023cc63ccfb None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:24.393204 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024cdb5-f253-400f-a2b9-3023cc63ccfb None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:24.393692 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9024cdb5-f253-400f-a2b9-3023cc63ccfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:24.412187 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-9024cdb5-f253-400f-a2b9-3023cc63ccfb None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:24.413717 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-9024cdb5-f253-400f-a2b9-3023cc63ccfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:24.485116 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 357/721] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:24.523641 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d71f8f-f538-4819-a3f2-f5c3454f7bee None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:24.573142 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:24.652172 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d71f8f-f538-4819-a3f2-f5c3454f7bee None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:24.696059 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba23ed-1350-4b65-a322-8cd3dc27c713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:24.696935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba23ed-1350-4b65-a322-8cd3dc27c713 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:24.697206 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba23ed-1350-4b65-a322-8cd3dc27c713 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:24.697482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bba23ed-1350-4b65-a322-8cd3dc27c713 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:24.712309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bba23ed-1350-4b65-a322-8cd3dc27c713 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:24.720019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-6bba23ed-1350-4b65-a322-8cd3dc27c713 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:48:24.720019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:24.720019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:24.720019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:24.720019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:24.720019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:24.720019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:24.720019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:24.720019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:24.720019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:24.720019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:24.720019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:24.720019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:24.720019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:24.720019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:24.720019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:24.721001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:24.721001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:24.721001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:24.721001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:24.721001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:24.721001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:24.721001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:24.721001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:24.721001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:24.721001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:24.721001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:24.721001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:24.721001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:24.721001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:24.721001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:24.721001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:24.721841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:24.721841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:24.721841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:24.721841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:24.721841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:24.721841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:24.721841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:24.721841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:24.721841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:24.721841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:24.721841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:48:24.721841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:24.726042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 365/722] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:48:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:24.727368 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024cdb5-f253-400f-a2b9-3023cc63ccfb None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:24.858048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9024cdb5-f253-400f-a2b9-3023cc63ccfb None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:24.894298 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9455dab-7572-4bcf-88b5-f4128dc750de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:24.899371 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9455dab-7572-4bcf-88b5-f4128dc750de None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:24.899885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9455dab-7572-4bcf-88b5-f4128dc750de None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:24.900376 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9455dab-7572-4bcf-88b5-f4128dc750de None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:24.901369 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9455dab-7572-4bcf-88b5-f4128dc750de None admin] RBAC: Adding query filter params (name=admin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:24.901888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9455dab-7572-4bcf-88b5-f4128dc750de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:24.906618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:24.907061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9455dab-7572-4bcf-88b5-f4128dc750de None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:24.917412 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 358/723] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:48:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:24.923663 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba23ed-1350-4b65-a322-8cd3dc27c713 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:25.089377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:25.098277 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bba23ed-1350-4b65-a322-8cd3dc27c713 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:25.152737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62978c03-9c94-4e0b-b785-b91ca8b25ba2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:25.154075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62978c03-9c94-4e0b-b785-b91ca8b25ba2 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:25.154540 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62978c03-9c94-4e0b-b785-b91ca8b25ba2 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:25.155010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62978c03-9c94-4e0b-b785-b91ca8b25ba2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:25.171639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62978c03-9c94-4e0b-b785-b91ca8b25ba2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:25.173439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62978c03-9c94-4e0b-b785-b91ca8b25ba2 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:25.191508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-62978c03-9c94-4e0b-b785-b91ca8b25ba2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:25.191508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:25.191508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:25.191508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:25.191508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:25.191508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:25.191508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:25.191508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:25.191508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:25.191508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:25.191508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:25.191508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:25.191508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:25.191508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:25.191508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:25.192406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:25.192406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:25.192406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:25.192406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:25.192406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:25.192406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:25.192406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:25.192406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:25.192406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:25.192406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:25.192406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:25.192406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:25.192406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:25.192406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:25.192406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:25.192406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:25.192406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:25.193470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:25.193470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:25.193470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:25.193470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:25.195721 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 366/724] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:25.203031 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9455dab-7572-4bcf-88b5-f4128dc750de None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:25.331008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9455dab-7572-4bcf-88b5-f4128dc750de None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:25.369587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a7bd4-93ad-49b4-9884-02838f4c5723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:25.370273 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a7bd4-93ad-49b4-9884-02838f4c5723 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:25.370563 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a7bd4-93ad-49b4-9884-02838f4c5723 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:25.370868 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918a7bd4-93ad-49b4-9884-02838f4c5723 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:25.372683 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b2df8-87f6-4e27-948d-28a7377cd5fb None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:25.372977 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b2df8-87f6-4e27-948d-28a7377cd5fb None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:25.373331 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2b2df8-87f6-4e27-948d-28a7377cd5fb None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:25.374823 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 367/725] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:48:25.375506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a7bd4-93ad-49b4-9884-02838f4c5723 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:25.375827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a7bd4-93ad-49b4-9884-02838f4c5723 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:25.377284 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918a7bd4-93ad-49b4-9884-02838f4c5723 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:25.386769 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded9548-e3c8-4d91-bbdc-4d75851f3cbd None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:25.387296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded9548-e3c8-4d91-bbdc-4d75851f3cbd None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:25.387495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aded9548-e3c8-4d91-bbdc-4d75851f3cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:25.391232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 359/726] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:25.402888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a7bd4-93ad-49b4-9884-02838f4c5723 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:25.430167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-aded9548-e3c8-4d91-bbdc-4d75851f3cbd None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:25.431607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-aded9548-e3c8-4d91-bbdc-4d75851f3cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:25.458450 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:25.510514 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918a7bd4-93ad-49b4-9884-02838f4c5723 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:25.514806 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 368/727] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:25.516254 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7ebe2-33f7-4d3f-9793-fabe06ac1433 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:25.516547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7ebe2-33f7-4d3f-9793-fabe06ac1433 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:25.516827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb7ebe2-33f7-4d3f-9793-fabe06ac1433 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:25.518474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 369/728] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:25.530218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074885e-a95c-4649-92d0-a9c34cde3b96 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:25.530511 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074885e-a95c-4649-92d0-a9c34cde3b96 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:25.530843 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3074885e-a95c-4649-92d0-a9c34cde3b96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:25.548160 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba2522-f7af-436e-9dc7-94230423b2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:25.549036 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba2522-f7af-436e-9dc7-94230423b2f5 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:25.549330 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba2522-f7af-436e-9dc7-94230423b2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:25.549613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ba2522-f7af-436e-9dc7-94230423b2f5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:25.557781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ba2522-f7af-436e-9dc7-94230423b2f5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:25.559139 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-3074885e-a95c-4649-92d0-a9c34cde3b96 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:25.559547 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ba2522-f7af-436e-9dc7-94230423b2f5 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:25.560400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-3074885e-a95c-4649-92d0-a9c34cde3b96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:25.577406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-a6ba2522-f7af-436e-9dc7-94230423b2f5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 22:48:25.577406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:25.577406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:25.577406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:25.577406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:25.577406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:25.577406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:25.577406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:25.577406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:25.577406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:25.577406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:25.577406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:25.577406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:25.577406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:25.577406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:25.577406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:25.580154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:25.580154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:25.580154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:25.580154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:25.580154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:25.580154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:25.580154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:25.580154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:25.580154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:25.580154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:25.580154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:25.580154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:25.580154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:25.580154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:25.580154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:25.580154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:25.580154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:25.580154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:25.582418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:25.582418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:25.582418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:25.582418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:25.582418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:25.582418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:25.582418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:25.582418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:25.582418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 22:48:25.582418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:25.582418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 360/729] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:48:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:25.582418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba2522-f7af-436e-9dc7-94230423b2f5 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:25.646821 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 370/730] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 22:48:25.647750 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074885e-a95c-4649-92d0-a9c34cde3b96 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:25.667535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:25.691541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ba2522-f7af-436e-9dc7-94230423b2f5 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:25.707172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:25.731822 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041430a6-ea03-4dc1-9c61-90184ff6abab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:25.733048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041430a6-ea03-4dc1-9c61-90184ff6abab None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:25.733490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041430a6-ea03-4dc1-9c61-90184ff6abab None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:25.733917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041430a6-ea03-4dc1-9c61-90184ff6abab None admin] PATH_INFO: `/v3/roles/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:25.745412 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041430a6-ea03-4dc1-9c61-90184ff6abab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:25.754124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-041430a6-ea03-4dc1-9c61-90184ff6abab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:25.754124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:25.754124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:25.754124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:25.754124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:25.754124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:25.754124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:25.754124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:25.754124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:25.754124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:25.754124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:25.754124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:25.754124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:25.754124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:25.754124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:25.754124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:25.755880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:25.755880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:25.755880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:25.755880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:25.755880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:25.755880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:25.755880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:25.755880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:25.755880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:25.755880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:25.755880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:25.755880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:25.755880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:25.755880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:25.755880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:25.755880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:25.756917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:25.756917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:25.756917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:25.756917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:25.756917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:25.756917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:25.756917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:25.756917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:25.756917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:25.756917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:25.756917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:25.756917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:25.759273 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 361/731] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:25.760199 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041430a6-ea03-4dc1-9c61-90184ff6abab None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:25.777782 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3074885e-a95c-4649-92d0-a9c34cde3b96 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:25.814525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a70fa-f6f0-4632-8e9c-08262890a441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:25.815240 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a70fa-f6f0-4632-8e9c-08262890a441 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:25.815526 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a70fa-f6f0-4632-8e9c-08262890a441 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:25.815806 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8a70fa-f6f0-4632-8e9c-08262890a441 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:25.816750 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a70fa-f6f0-4632-8e9c-08262890a441 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:25.817322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a70fa-f6f0-4632-8e9c-08262890a441 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:25.819164 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8a70fa-f6f0-4632-8e9c-08262890a441 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:25.837624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 371/732] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jun 16 22:48:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:25.838532 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a70fa-f6f0-4632-8e9c-08262890a441 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:25.877391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041430a6-ea03-4dc1-9c61-90184ff6abab None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:25.908634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:25.916493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bac33-26d7-454c-b7e6-705960f05cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:25.917514 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bac33-26d7-454c-b7e6-705960f05cca None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:25.918052 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bac33-26d7-454c-b7e6-705960f05cca None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:25.918502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6bac33-26d7-454c-b7e6-705960f05cca None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:25.930098 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6bac33-26d7-454c-b7e6-705960f05cca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:25.938008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-9d6bac33-26d7-454c-b7e6-705960f05cca None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:48:25.938008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:25.938008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:25.938008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:25.938008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:25.938008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:25.938008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:25.938008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:25.938008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:25.938008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:25.938008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:25.938008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:25.938008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:25.938008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:25.938008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:25.938008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:25.939140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:25.939140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:25.939140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:25.939140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:25.939140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:25.939140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:25.939140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:25.939140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:25.939140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:25.939140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:25.939140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:25.939140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:25.939140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:25.939140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:25.939140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:25.939140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:25.940118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:25.940118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:25.940118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:25.940118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:25.940118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:25.940118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:25.940118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:25.940118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:25.940118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:25.940118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:25.940118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:48:25.940118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:25.942285 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 362/733] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:48:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:25.943281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bac33-26d7-454c-b7e6-705960f05cca None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:25.948613 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8a70fa-f6f0-4632-8e9c-08262890a441 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:25.976984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:25.988359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff1b48-0eb2-49c4-ab50-ca137117fe3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:25.989044 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff1b48-0eb2-49c4-ab50-ca137117fe3e None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:25.989863 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff1b48-0eb2-49c4-ab50-ca137117fe3e None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:25.990150 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ff1b48-0eb2-49c4-ab50-ca137117fe3e None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:25.990950 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff1b48-0eb2-49c4-ab50-ca137117fe3e None admin] RBAC: Adding query filter params (name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:25.991263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff1b48-0eb2-49c4-ab50-ca137117fe3e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:25.992748 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ff1b48-0eb2-49c4-ab50-ca137117fe3e None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:26.006495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 372/734] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:48:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:26.007554 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff1b48-0eb2-49c4-ab50-ca137117fe3e None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:26.054734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6bac33-26d7-454c-b7e6-705960f05cca None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:26.090854 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ab187d-714e-4b91-943d-609e6ea01989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:26.091820 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab187d-714e-4b91-943d-609e6ea01989 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:26.092290 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab187d-714e-4b91-943d-609e6ea01989 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:26.092770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ab187d-714e-4b91-943d-609e6ea01989 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:26.123241 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ff1b48-0eb2-49c4-ab50-ca137117fe3e None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:26.171414 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:26.173609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab187d-714e-4b91-943d-609e6ea01989 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:26.175570 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ab187d-714e-4b91-943d-609e6ea01989 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:26.187003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa498a23-48f4-41af-b3cb-686640bd1fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:26.189933 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa498a23-48f4-41af-b3cb-686640bd1fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:26.189933 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa498a23-48f4-41af-b3cb-686640bd1fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:26.190370 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa498a23-48f4-41af-b3cb-686640bd1fb4 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:26.191200 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa498a23-48f4-41af-b3cb-686640bd1fb4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:26.191522 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa498a23-48f4-41af-b3cb-686640bd1fb4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:26.193560 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa498a23-48f4-41af-b3cb-686640bd1fb4 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:26.207293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 373/735] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:48:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:26.207293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa498a23-48f4-41af-b3cb-686640bd1fb4 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:26.231458 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:26.314542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 363/736] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:48:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 22:48:26.314915 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0938e6-54c1-441d-ab32-94f220727c67 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:26.315217 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0938e6-54c1-441d-ab32-94f220727c67 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:26.322626 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0938e6-54c1-441d-ab32-94f220727c67 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:26.322626 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 364/737] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:26.322626 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0938e6-54c1-441d-ab32-94f220727c67 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:26.365963 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa498a23-48f4-41af-b3cb-686640bd1fb4 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:26.412965 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e69de-029d-4312-b719-ffaabb068078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:26.414086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e69de-029d-4312-b719-ffaabb068078 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:26.414479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e69de-029d-4312-b719-ffaabb068078 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:26.414894 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141e69de-029d-4312-b719-ffaabb068078 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:26.425991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e69de-029d-4312-b719-ffaabb068078 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:26.428195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141e69de-029d-4312-b719-ffaabb068078 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:26.434202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0938e6-54c1-441d-ab32-94f220727c67 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:26.452388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-141e69de-029d-4312-b719-ffaabb068078 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:26.452388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:26.452388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:26.452388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:26.452388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:26.452388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:26.452388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:26.452388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:26.452388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:26.452388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:26.452388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:26.452388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:26.452388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:26.452388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:26.452388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:26.453272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:26.453272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:26.453272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:26.453272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:26.453272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:26.453272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:26.453272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:26.453272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:26.453272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:26.453272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:26.453272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:26.453272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:26.453272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:26.453272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:26.453272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:26.453272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:26.453272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:26.454167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:26.454167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:26.454167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:26.454167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:26.458243 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 374/738] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:26.459109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e69de-029d-4312-b719-ffaabb068078 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:26.465478 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:26.471797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e16f5c7-90ef-4123-b1cc-7fecf92bba54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:26.472550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16f5c7-90ef-4123-b1cc-7fecf92bba54 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:26.472824 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16f5c7-90ef-4123-b1cc-7fecf92bba54 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:26.473106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e16f5c7-90ef-4123-b1cc-7fecf92bba54 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:26.483830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16f5c7-90ef-4123-b1cc-7fecf92bba54 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:26.485266 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e16f5c7-90ef-4123-b1cc-7fecf92bba54 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:26.503394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-8e16f5c7-90ef-4123-b1cc-7fecf92bba54 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:26.503394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:26.503394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:26.503394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:26.503394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:26.503394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:26.503394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:26.503394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:26.503394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:26.503394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:26.503394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:26.503394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:26.503394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:26.503394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:26.503394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:26.504264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:26.504264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:26.504264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:26.504264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:26.504264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:26.504264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:26.504264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:26.504264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:26.504264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:26.504264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:26.504264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:26.504264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:26.504264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:26.504264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:26.504264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:26.504264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:26.504264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:26.505164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:26.505164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:26.505164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:26.505164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:26.505164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 365/739] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:26.505429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8226e3-95c3-4079-a3d3-932fe5601e55 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:26.505939 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8226e3-95c3-4079-a3d3-932fe5601e55 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:26.505939 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8226e3-95c3-4079-a3d3-932fe5601e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:26.517264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:26.522807 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-6c8226e3-95c3-4079-a3d3-932fe5601e55 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:26.523991 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6c8226e3-95c3-4079-a3d3-932fe5601e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:26.582053 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141e69de-029d-4312-b719-ffaabb068078 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:26.597221 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 366/740] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:26.598059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8226e3-95c3-4079-a3d3-932fe5601e55 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:26.628160 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4d710-907d-46c7-bd1a-7eb8237788c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:26.629104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4d710-907d-46c7-bd1a-7eb8237788c5 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:26.629615 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4d710-907d-46c7-bd1a-7eb8237788c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:26.630353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf4d710-907d-46c7-bd1a-7eb8237788c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:26.641431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf4d710-907d-46c7-bd1a-7eb8237788c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:26.642998 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf4d710-907d-46c7-bd1a-7eb8237788c5 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:26.663440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-0cf4d710-907d-46c7-bd1a-7eb8237788c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:26.663440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:26.663440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:26.663440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:26.663440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:26.663440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:26.663440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:26.663440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:26.663440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:26.663440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:26.663440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:26.663440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:26.663440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:26.663440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:26.663440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:26.664586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:26.664586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:26.664586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:26.664586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:26.664586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:26.664586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:26.664586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:26.664586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:26.664586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:26.664586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:26.664586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:26.664586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:26.664586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:26.664586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:26.664586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:26.664586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:26.664586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:26.665698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:26.665698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:26.665698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:26.665698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:26.671185 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 375/741] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:26.671185 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4d710-907d-46c7-bd1a-7eb8237788c5 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:26.720906 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8226e3-95c3-4079-a3d3-932fe5601e55 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:26.735672 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:26.757230 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec205f13-3d34-4e42-81ef-e1350fae0cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:26.757896 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec205f13-3d34-4e42-81ef-e1350fae0cda None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:26.758187 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec205f13-3d34-4e42-81ef-e1350fae0cda None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:26.758469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec205f13-3d34-4e42-81ef-e1350fae0cda None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:26.759236 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec205f13-3d34-4e42-81ef-e1350fae0cda None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:26.759552 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec205f13-3d34-4e42-81ef-e1350fae0cda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:26.760933 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec205f13-3d34-4e42-81ef-e1350fae0cda None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:26.771883 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 367/742] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:26.772629 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec205f13-3d34-4e42-81ef-e1350fae0cda None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:26.841805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:26.848603 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf4d710-907d-46c7-bd1a-7eb8237788c5 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:26.873457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec205f13-3d34-4e42-81ef-e1350fae0cda None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:26.882100 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f83b0-5fcb-40f6-867b-03c788c4d365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:26.882815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f83b0-5fcb-40f6-867b-03c788c4d365 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:26.883108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f83b0-5fcb-40f6-867b-03c788c4d365 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:26.883388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a30f83b0-5fcb-40f6-867b-03c788c4d365 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:26.884101 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f83b0-5fcb-40f6-867b-03c788c4d365 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:26.884410 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f83b0-5fcb-40f6-867b-03c788c4d365 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:26.885764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a30f83b0-5fcb-40f6-867b-03c788c4d365 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:26.897304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 376/743] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:26.898114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f83b0-5fcb-40f6-867b-03c788c4d365 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:26.908254 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b0af1-e04a-40b5-be00-2b26be43142f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:26.908924 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b0af1-e04a-40b5-be00-2b26be43142f None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:26.909217 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b0af1-e04a-40b5-be00-2b26be43142f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:26.909501 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b0af1-e04a-40b5-be00-2b26be43142f None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:26.910223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b0af1-e04a-40b5-be00-2b26be43142f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:26.911716 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b0af1-e04a-40b5-be00-2b26be43142f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:26.955279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 368/744] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 22:48:26 2022] POST /identity/v3/services => generated 251 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:48:26.956600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8917068-77af-4fd7-9228-b47463d2d4e0 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:26.957003 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8917068-77af-4fd7-9228-b47463d2d4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:26.957186 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8917068-77af-4fd7-9228-b47463d2d4e0 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:26.958782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 369/745] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:26.959487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8917068-77af-4fd7-9228-b47463d2d4e0 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:27.009509 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:27.040665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a30f83b0-5fcb-40f6-867b-03c788c4d365 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:27.103390 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488066e-4160-45e3-a903-175370b9b8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:27.104444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488066e-4160-45e3-a903-175370b9b8f6 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:27.104894 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488066e-4160-45e3-a903-175370b9b8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:27.105170 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488066e-4160-45e3-a903-175370b9b8f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:27.105914 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b488066e-4160-45e3-a903-175370b9b8f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:27.106230 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b488066e-4160-45e3-a903-175370b9b8f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:27.107654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b488066e-4160-45e3-a903-175370b9b8f6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:27.110913 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8917068-77af-4fd7-9228-b47463d2d4e0 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:27.121793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 377/746] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:27.122091 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488066e-4160-45e3-a903-175370b9b8f6 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:27.137038 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:27.152787 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc02d7d-f5b6-4590-8377-338a348b5494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:27.154108 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc02d7d-f5b6-4590-8377-338a348b5494 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:27.154402 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc02d7d-f5b6-4590-8377-338a348b5494 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:27.154713 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc02d7d-f5b6-4590-8377-338a348b5494 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:27.163122 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc02d7d-f5b6-4590-8377-338a348b5494 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:27.164568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc02d7d-f5b6-4590-8377-338a348b5494 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:27.179613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-acc02d7d-f5b6-4590-8377-338a348b5494 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 22:48:27.179613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:27.179613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:27.179613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:27.179613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:27.179613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:27.179613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:27.179613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:27.179613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:27.179613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:27.179613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:27.179613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:27.179613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:27.179613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:27.179613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:27.179613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:27.180627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:27.180627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:27.180627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:27.180627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:27.180627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:27.180627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:27.180627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:27.180627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:27.180627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:27.180627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:27.180627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:27.180627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:27.180627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:27.180627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:27.180627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:27.180627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:27.180627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:27.180627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:27.181576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:27.181576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:27.181576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:27.181576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:27.181576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:27.181576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:27.181576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:27.181576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:27.181576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 22:48:27.181576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:27.181576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 370/747] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 22:48:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:27.182384 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c2e92-f17c-4c30-895e-ec4eccb30cf1 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:27.182708 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c2e92-f17c-4c30-895e-ec4eccb30cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:27.182953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29c2e92-f17c-4c30-895e-ec4eccb30cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:27.203183 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-d29c2e92-f17c-4c30-895e-ec4eccb30cf1 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:27.204389 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d29c2e92-f17c-4c30-895e-ec4eccb30cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:27.241501 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488066e-4160-45e3-a903-175370b9b8f6 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:27.278767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbde89d-f7a5-4a9a-b594-84148265fd3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:27.279882 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbde89d-f7a5-4a9a-b594-84148265fd3e None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:27.280175 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbde89d-f7a5-4a9a-b594-84148265fd3e None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:27.280454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbde89d-f7a5-4a9a-b594-84148265fd3e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:27.291917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 371/748] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:27 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 22:48:27.291917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c2e92-f17c-4c30-895e-ec4eccb30cf1 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:27.292171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbde89d-f7a5-4a9a-b594-84148265fd3e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:27.292171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbde89d-f7a5-4a9a-b594-84148265fd3e None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:27.306973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-ecbde89d-f7a5-4a9a-b594-84148265fd3e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 22:48:27.306973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:27.306973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:27.306973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:27.306973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:27.306973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:27.306973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:27.306973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:27.306973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:27.306973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:27.306973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:27.306973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:27.306973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:27.306973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:27.306973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:27.306973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:27.308191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:27.308191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:27.308191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:27.308191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:27.308191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:27.308191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:27.308191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:27.308191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:27.308191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:27.308191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:27.308191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:27.308191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:27.308191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:27.308191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:27.308191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:27.308191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:27.308191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:27.308191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:27.309345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:27.309345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:27.309345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:27.309345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:27.309345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:27.309345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:27.309345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:27.309345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:27.309345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 22:48:27.309345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:27.309345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 378/749] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jun 16 22:48:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:27.310047 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbde89d-f7a5-4a9a-b594-84148265fd3e None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:27.323312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:27.382415 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:27.413996 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29c2e92-f17c-4c30-895e-ec4eccb30cf1 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:27.415546 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbde89d-f7a5-4a9a-b594-84148265fd3e None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:27.450539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7742b2-b09a-40e5-8dd0-8e3f66564bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:27.451371 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7742b2-b09a-40e5-8dd0-8e3f66564bbb None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:27.451654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7742b2-b09a-40e5-8dd0-8e3f66564bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:27.451982 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7742b2-b09a-40e5-8dd0-8e3f66564bbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:27.454855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0f82c-cca6-49db-8ad9-a74808cf351e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:27.455534 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0f82c-cca6-49db-8ad9-a74808cf351e None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:27.455826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0f82c-cca6-49db-8ad9-a74808cf351e None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:27.456105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de0f82c-cca6-49db-8ad9-a74808cf351e None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:27.456942 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de0f82c-cca6-49db-8ad9-a74808cf351e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:27.457254 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de0f82c-cca6-49db-8ad9-a74808cf351e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:27.459232 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de0f82c-cca6-49db-8ad9-a74808cf351e None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:27.459657 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7742b2-b09a-40e5-8dd0-8e3f66564bbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:27.461107 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7742b2-b09a-40e5-8dd0-8e3f66564bbb None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:27.475481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-7e7742b2-b09a-40e5-8dd0-8e3f66564bbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:27.475481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:27.475481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:27.475481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:27.475481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:27.475481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:27.475481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:27.475481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:27.475481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:27.475481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:27.475481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:27.475481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:27.475481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:27.475481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:27.475481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:27.476608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:27.476608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:27.476608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:27.476608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:27.476608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:27.476608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:27.476608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:27.476608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:27.476608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:27.476608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:27.476608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:27.476608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:27.476608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:27.476608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:27.476608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:27.476608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:27.476608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:27.477523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:27.477523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:27.477523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:27.477523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:27.477523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:27.477523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:27.477523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:27.477523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:27.477523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 372/750] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:27.477523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7742b2-b09a-40e5-8dd0-8e3f66564bbb None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:27.488119 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 379/751] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jun 16 22:48:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:27.488119 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0f82c-cca6-49db-8ad9-a74808cf351e None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:27.572506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:27.590706 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7742b2-b09a-40e5-8dd0-8e3f66564bbb None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:27.596504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de0f82c-cca6-49db-8ad9-a74808cf351e None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:27.625396 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:27.634670 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a05da2-d9a6-4d83-9193-e775eeeda833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:27.635367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a05da2-d9a6-4d83-9193-e775eeeda833 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:27.635655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a05da2-d9a6-4d83-9193-e775eeeda833 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:27.635933 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a05da2-d9a6-4d83-9193-e775eeeda833 None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:27.636777 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a05da2-d9a6-4d83-9193-e775eeeda833 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:27.637010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757d2ba-5928-4591-a6d7-ba1886da45d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:27.637122 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a05da2-d9a6-4d83-9193-e775eeeda833 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:27.637738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757d2ba-5928-4591-a6d7-ba1886da45d6 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:27.638030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757d2ba-5928-4591-a6d7-ba1886da45d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:27.638314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f757d2ba-5928-4591-a6d7-ba1886da45d6 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:27.638517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a05da2-d9a6-4d83-9193-e775eeeda833 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:27.639069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757d2ba-5928-4591-a6d7-ba1886da45d6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:27.640416 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f757d2ba-5928-4591-a6d7-ba1886da45d6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:27.663574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 373/752] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jun 16 22:48:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:27.663574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a05da2-d9a6-4d83-9193-e775eeeda833 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:27.663866 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 380/753] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 22:48:27 2022] POST /identity/v3/services => generated 240 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:48:27.663866 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757d2ba-5928-4591-a6d7-ba1886da45d6 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:27.801571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a05da2-d9a6-4d83-9193-e775eeeda833 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:27.833920 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f757d2ba-5928-4591-a6d7-ba1886da45d6 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:27.845532 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679acaa1-7982-4d6c-b905-848e1bfcad9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:27.846234 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679acaa1-7982-4d6c-b905-848e1bfcad9d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:27.846572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679acaa1-7982-4d6c-b905-848e1bfcad9d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:27.846928 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679acaa1-7982-4d6c-b905-848e1bfcad9d None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:27.847731 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679acaa1-7982-4d6c-b905-848e1bfcad9d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:27.848052 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679acaa1-7982-4d6c-b905-848e1bfcad9d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:27.850398 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679acaa1-7982-4d6c-b905-848e1bfcad9d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:27.866191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 374/754] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:27.870711 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679acaa1-7982-4d6c-b905-848e1bfcad9d None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:27.874674 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:27.887740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb3730-2871-4997-9345-738aa931f4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:27.888394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb3730-2871-4997-9345-738aa931f4ee None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:27.888678 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb3730-2871-4997-9345-738aa931f4ee None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:27.888952 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bb3730-2871-4997-9345-738aa931f4ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:27.957526 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:27.983616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb3730-2871-4997-9345-738aa931f4ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:27.985642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bb3730-2871-4997-9345-738aa931f4ee None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:28.000045 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679acaa1-7982-4d6c-b905-848e1bfcad9d None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:28.036309 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be2065-875b-4f9f-92ef-5fb535c5a329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:28.037278 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be2065-875b-4f9f-92ef-5fb535c5a329 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:28.037667 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be2065-875b-4f9f-92ef-5fb535c5a329 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:28.038048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be2065-875b-4f9f-92ef-5fb535c5a329 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:28.049437 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be2065-875b-4f9f-92ef-5fb535c5a329 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:28.051129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be2065-875b-4f9f-92ef-5fb535c5a329 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:28.069668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-27be2065-875b-4f9f-92ef-5fb535c5a329 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:28.069668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:28.069668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:28.069668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:28.069668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:28.069668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:28.069668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:28.069668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:28.069668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:28.069668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:28.069668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:28.069668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:28.069668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:28.069668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:28.069668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:28.070791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:28.070791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:28.070791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:28.070791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:28.070791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:28.070791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:28.070791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:28.070791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:28.070791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:28.070791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:28.070791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:28.070791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:28.070791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:28.070791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:28.070791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:28.070791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:28.070791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:28.071854 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:28.071854 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:28.071854 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:28.071854 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:28.077857 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 375/755] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:28.078836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be2065-875b-4f9f-92ef-5fb535c5a329 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:28.114977 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 381/756] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:48:27 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:48:28.116241 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb3730-2871-4997-9345-738aa931f4ee None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:28.140826 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:28.190622 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be2065-875b-4f9f-92ef-5fb535c5a329 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:28.206959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:28.227593 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79fdff-a3e7-46a5-a5c8-86dc68e7c5ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:28.228418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79fdff-a3e7-46a5-a5c8-86dc68e7c5ba None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:28.228813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79fdff-a3e7-46a5-a5c8-86dc68e7c5ba None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:28.229197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b79fdff-a3e7-46a5-a5c8-86dc68e7c5ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:28.229793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bb3730-2871-4997-9345-738aa931f4ee None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:28.230236 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79fdff-a3e7-46a5-a5c8-86dc68e7c5ba None admin] RBAC: Adding query filter params (role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=bef518c9430e4274b1066241f821f4a1) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:28.230698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79fdff-a3e7-46a5-a5c8-86dc68e7c5ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:28.232242 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b79fdff-a3e7-46a5-a5c8-86dc68e7c5ba None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:28.242981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 376/757] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:48:28 2022] GET /identity/v3/role_assignments?user.id=bef518c9430e4274b1066241f821f4a1&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:28.243998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79fdff-a3e7-46a5-a5c8-86dc68e7c5ba None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:28.278485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289dca1d-f115-49e9-bf21-34c967b0d3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:28.279977 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289dca1d-f115-49e9-bf21-34c967b0d3eb None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:28.280629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289dca1d-f115-49e9-bf21-34c967b0d3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:28.282582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289dca1d-f115-49e9-bf21-34c967b0d3eb None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:28.283642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289dca1d-f115-49e9-bf21-34c967b0d3eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:28.285009 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289dca1d-f115-49e9-bf21-34c967b0d3eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:28.289372 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289dca1d-f115-49e9-bf21-34c967b0d3eb None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:28.312485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 382/758] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:28.319274 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289dca1d-f115-49e9-bf21-34c967b0d3eb None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:28.433786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b79fdff-a3e7-46a5-a5c8-86dc68e7c5ba None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:28.456467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:28.484539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580b1c7-0ac7-409c-9bdd-3af87b1e93a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:28.485585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580b1c7-0ac7-409c-9bdd-3af87b1e93a0 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:28.486056 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580b1c7-0ac7-409c-9bdd-3af87b1e93a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:28.486431 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8580b1c7-0ac7-409c-9bdd-3af87b1e93a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:28.517314 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580b1c7-0ac7-409c-9bdd-3af87b1e93a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:28.519140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8580b1c7-0ac7-409c-9bdd-3af87b1e93a0 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:28.524844 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289dca1d-f115-49e9-bf21-34c967b0d3eb None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:28.558618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-8580b1c7-0ac7-409c-9bdd-3af87b1e93a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:28.558618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:28.558618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:28.558618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:28.558618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:28.558618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:28.558618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:28.558618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:28.558618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:28.558618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:28.558618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:28.558618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:28.558618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:28.558618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:28.558618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:28.559562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:28.559562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:28.559562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:28.559562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:28.559562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:28.559562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:28.559562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:28.559562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:28.559562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:28.559562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:28.559562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:28.559562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:28.559562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:28.559562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:28.559562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:28.559562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:28.559562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:28.560441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:28.560441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:28.560441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:28.560441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:28.560441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 377/759] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 310 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:28.560441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580b1c7-0ac7-409c-9bdd-3af87b1e93a0 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:28.601151 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6fae0-1ef9-4316-ae30-1e95f912d9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:28.602030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6fae0-1ef9-4316-ae30-1e95f912d9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:28.602325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6fae0-1ef9-4316-ae30-1e95f912d9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:28.602674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e6fae0-1ef9-4316-ae30-1e95f912d9f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:28.618621 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e6fae0-1ef9-4316-ae30-1e95f912d9f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:28.620099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e6fae0-1ef9-4316-ae30-1e95f912d9f0 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:28.653428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-b2e6fae0-1ef9-4316-ae30-1e95f912d9f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:28.653428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:28.653428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:28.653428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:28.653428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:28.653428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:28.653428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:28.653428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:28.653428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:28.653428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:28.653428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:28.653428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:28.653428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:28.653428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:28.653428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:28.654374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:28.654374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:28.654374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:28.654374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:28.654374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:28.654374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:28.654374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:28.654374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:28.654374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:28.654374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:28.654374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:28.654374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:28.654374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:28.654374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:28.654374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:28.654374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:28.654374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:28.655276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:28.655276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:28.655276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:28.655276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:28.655276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:28.655276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:28.655276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:28.655276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:28.662506 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 383/760] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:28.665378 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6fae0-1ef9-4316-ae30-1e95f912d9f0 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:28.670378 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:28.779690 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8580b1c7-0ac7-409c-9bdd-3af87b1e93a0 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:28.830630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e6fae0-1ef9-4316-ae30-1e95f912d9f0 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:28.861437 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01019e1-8dce-45c9-b23b-5f8341ec58c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:28.862135 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01019e1-8dce-45c9-b23b-5f8341ec58c9 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:28.862420 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01019e1-8dce-45c9-b23b-5f8341ec58c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:28.862727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01019e1-8dce-45c9-b23b-5f8341ec58c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:28.863485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01019e1-8dce-45c9-b23b-5f8341ec58c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:28.863799 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01019e1-8dce-45c9-b23b-5f8341ec58c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:28.872009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01019e1-8dce-45c9-b23b-5f8341ec58c9 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:28.904654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d83405-f0f2-4da5-a82d-910ff42c93a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:28.905352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d83405-f0f2-4da5-a82d-910ff42c93a2 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:28.905644 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d83405-f0f2-4da5-a82d-910ff42c93a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:28.905923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d83405-f0f2-4da5-a82d-910ff42c93a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:28.906767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d83405-f0f2-4da5-a82d-910ff42c93a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:28.907078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d83405-f0f2-4da5-a82d-910ff42c93a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:28.907299 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 378/761] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:28.908716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d83405-f0f2-4da5-a82d-910ff42c93a2 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:28.920130 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01019e1-8dce-45c9-b23b-5f8341ec58c9 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:28.937136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 384/762] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:28.942761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d83405-f0f2-4da5-a82d-910ff42c93a2 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:28.954524 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:29.045204 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01019e1-8dce-45c9-b23b-5f8341ec58c9 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:29.088239 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0ee26-77be-4300-872e-a80e8ff6a89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:29.091147 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0ee26-77be-4300-872e-a80e8ff6a89a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:29.091429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0ee26-77be-4300-872e-a80e8ff6a89a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:29.091707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb0ee26-77be-4300-872e-a80e8ff6a89a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:29.107553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0ee26-77be-4300-872e-a80e8ff6a89a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:29.109004 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb0ee26-77be-4300-872e-a80e8ff6a89a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:29.115365 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:29.146394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-3cb0ee26-77be-4300-872e-a80e8ff6a89a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:29.146394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:29.146394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:29.146394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:29.146394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:29.146394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:29.146394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:29.146394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:29.146394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:29.146394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:29.146394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:29.146394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:29.146394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:29.146394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:29.146394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:29.147414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:29.147414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:29.147414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:29.147414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:29.147414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:29.147414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:29.147414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:29.147414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:29.147414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:29.147414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:29.147414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:29.147414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:29.147414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:29.147414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:29.147414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:29.147414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:29.147414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:29.148276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:29.148276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:29.148276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:29.148276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:29.148276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:29.148276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:29.148276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:29.148276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:29.150890 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 379/763] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:29.159234 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0ee26-77be-4300-872e-a80e8ff6a89a None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:29.201235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d83405-f0f2-4da5-a82d-910ff42c93a2 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:29.252774 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75671a-e0e8-4c96-91d9-fef133d1a7b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:29.253681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75671a-e0e8-4c96-91d9-fef133d1a7b6 None admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:29.253964 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75671a-e0e8-4c96-91d9-fef133d1a7b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:29.254245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75671a-e0e8-4c96-91d9-fef133d1a7b6 None admin] PATH_INFO: `/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/4180f2593149437fa27effd8d1338d10/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:29.261869 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75671a-e0e8-4c96-91d9-fef133d1a7b6 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc48914270df4db996bd2a4b5b9fc777, user_id=4180f2593149437fa27effd8d1338d10, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:29.263329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef75671a-e0e8-4c96-91d9-fef133d1a7b6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:29.285440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 385/764] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:48:28 2022] PUT /identity/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/4180f2593149437fa27effd8d1338d10/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:48:29.287437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceed88f-6e74-48f7-9cb1-1e296dd1512a None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:29.287726 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceed88f-6e74-48f7-9cb1-1e296dd1512a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:29.288002 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceed88f-6e74-48f7-9cb1-1e296dd1512a None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:29.289750 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 386/765] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:29.306143 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc4b70-4b59-4e1d-8fcb-b071521bd860 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:29.306443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc4b70-4b59-4e1d-8fcb-b071521bd860 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:29.306639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fc4b70-4b59-4e1d-8fcb-b071521bd860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:29.340825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-72fc4b70-4b59-4e1d-8fcb-b071521bd860 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:29.342425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-72fc4b70-4b59-4e1d-8fcb-b071521bd860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:29.344930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:29.355005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb0ee26-77be-4300-872e-a80e8ff6a89a None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:29.394968 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c45123-5272-4cf4-864c-83b8aa2a06b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:29.396079 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c45123-5272-4cf4-864c-83b8aa2a06b2 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:29.396375 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c45123-5272-4cf4-864c-83b8aa2a06b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:29.396660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c45123-5272-4cf4-864c-83b8aa2a06b2 None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:29.397465 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c45123-5272-4cf4-864c-83b8aa2a06b2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:29.397781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c45123-5272-4cf4-864c-83b8aa2a06b2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:29.399286 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c45123-5272-4cf4-864c-83b8aa2a06b2 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:29.415354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 380/766] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:29.421916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c45123-5272-4cf4-864c-83b8aa2a06b2 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:29.492134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 387/767] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:29 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:48:29.518065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc4b70-4b59-4e1d-8fcb-b071521bd860 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:29.593023 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:29.634507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c45123-5272-4cf4-864c-83b8aa2a06b2 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:29.652130 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fc4b70-4b59-4e1d-8fcb-b071521bd860 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:29.707331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e25ced-b6c0-4f96-81e4-d0f927d2dfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:29.708699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e25ced-b6c0-4f96-81e4-d0f927d2dfb7 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:29.708988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e25ced-b6c0-4f96-81e4-d0f927d2dfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:29.709267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e25ced-b6c0-4f96-81e4-d0f927d2dfb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:29.710100 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e25ced-b6c0-4f96-81e4-d0f927d2dfb7 None admin] RBAC: Adding query filter params (role.id=ec841a43f9924fa1b2209bc5ecea5e04, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=faf778e713b54a638c1db4a5db6180b6) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:29.710415 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e25ced-b6c0-4f96-81e4-d0f927d2dfb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:29.711876 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e25ced-b6c0-4f96-81e4-d0f927d2dfb7 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:29.715562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba7874-f6e4-459a-a512-d06b83a3bf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:29.716381 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba7874-f6e4-459a-a512-d06b83a3bf25 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:29.716663 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba7874-f6e4-459a-a512-d06b83a3bf25 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:29.716937 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ba7874-f6e4-459a-a512-d06b83a3bf25 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:29.717597 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba7874-f6e4-459a-a512-d06b83a3bf25 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:29.718992 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ba7874-f6e4-459a-a512-d06b83a3bf25 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:29.726395 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 381/768] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:48:29 2022] GET /identity/v3/role_assignments?user.id=faf778e713b54a638c1db4a5db6180b6&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=ec841a43f9924fa1b2209bc5ecea5e04 => generated 259 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:29.735486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-92ba7874-f6e4-459a-a512-d06b83a3bf25 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 22:48:29.735486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:29.735486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:29.735486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:29.735486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:29.735486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:29.735486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:29.735486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:29.735486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 22:48:29.735486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:29.735486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:29.735486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:29.735486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:29.735486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:29.735486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:29.735486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:29.736520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:29.736520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:29.736520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:29.736520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:29.736520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:29.736520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:29.736520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:29.736520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:29.736520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:29.736520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:29.736520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:29.736520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:29.736520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:29.736520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:29.736520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:29.736520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:29.737329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:29.737329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:29.737329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:29.737329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:29.737329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:29.737329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:29.737329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:29.737329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:29.737329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:29.737329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:29.737329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:29.737329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:29.737329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:29.737329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 22:48:29.737329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:29.737329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 388/769] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jun 16 22:48:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:29.743780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e25ced-b6c0-4f96-81e4-d0f927d2dfb7 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:29.771379 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:29.902301 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e25ced-b6c0-4f96-81e4-d0f927d2dfb7 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:29.946323 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cc924c-b685-432c-bf71-379af55337c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:29.947047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc924c-b685-432c-bf71-379af55337c4 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:29.947471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc924c-b685-432c-bf71-379af55337c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:29.947752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cc924c-b685-432c-bf71-379af55337c4 None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:29.948469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc924c-b685-432c-bf71-379af55337c4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:29.948782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc924c-b685-432c-bf71-379af55337c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:29.950177 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33cc924c-b685-432c-bf71-379af55337c4 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:29.955306 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 382/770] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jun 16 22:48:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:29.961900 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba7874-f6e4-459a-a512-d06b83a3bf25 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:30.046332 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e9f32-7618-448c-94d5-12e3cabfaaa1 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:30.047005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e9f32-7618-448c-94d5-12e3cabfaaa1 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:30.047521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72e9f32-7618-448c-94d5-12e3cabfaaa1 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:30.049409 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 383/771] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:48:30.054506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34626a-2d21-4101-9ab1-30a2af0acc97 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:30.055088 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34626a-2d21-4101-9ab1-30a2af0acc97 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:30.055593 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de34626a-2d21-4101-9ab1-30a2af0acc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:30.088783 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-de34626a-2d21-4101-9ab1-30a2af0acc97 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:30.091930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-de34626a-2d21-4101-9ab1-30a2af0acc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:30.139410 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ba7874-f6e4-459a-a512-d06b83a3bf25 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:30.177343 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:30.199789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de220098-28fe-48c5-9e7f-86f1256e19a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:30.201297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de220098-28fe-48c5-9e7f-86f1256e19a7 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:30.201851 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de220098-28fe-48c5-9e7f-86f1256e19a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:30.202356 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de220098-28fe-48c5-9e7f-86f1256e19a7 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:30.203316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de220098-28fe-48c5-9e7f-86f1256e19a7 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:30.203832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de220098-28fe-48c5-9e7f-86f1256e19a7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:30.205430 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de220098-28fe-48c5-9e7f-86f1256e19a7 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:30.211514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 389/772] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jun 16 22:48:29 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:30.221993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de220098-28fe-48c5-9e7f-86f1256e19a7 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:30.242775 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 384/773] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:30 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:48:30.270566 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34626a-2d21-4101-9ab1-30a2af0acc97 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:30.284002 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:30.392377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de34626a-2d21-4101-9ab1-30a2af0acc97 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:30.421329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de220098-28fe-48c5-9e7f-86f1256e19a7 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:30.433980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472edf7-1eca-4bbf-95dc-f5843f912a93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:30.435107 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472edf7-1eca-4bbf-95dc-f5843f912a93 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:30.435619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472edf7-1eca-4bbf-95dc-f5843f912a93 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:30.436111 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2472edf7-1eca-4bbf-95dc-f5843f912a93 None admin] PATH_INFO: `/v3/services/network` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:30.436976 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472edf7-1eca-4bbf-95dc-f5843f912a93 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:30.438561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2472edf7-1eca-4bbf-95dc-f5843f912a93 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:30.452723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-2472edf7-1eca-4bbf-95dc-f5843f912a93 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 22:48:30.452723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:30.452723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:30.452723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:30.452723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:30.452723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:30.452723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:30.452723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:30.452723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 22:48:30.452723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:30.452723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:30.452723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:30.452723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:30.452723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:30.452723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:30.452723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:30.452723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:30.453728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:30.453728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:30.453728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:30.453728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:30.453728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:30.453728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:30.453728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:30.453728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:30.453728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:30.453728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:30.453728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:30.453728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:30.453728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:30.453728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:30.453728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:30.453728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:30.454562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:30.454562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:30.454562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:30.454562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:30.454562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:30.454562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:30.454562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:30.454562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:30.454562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:30.454562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:30.454562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:30.454562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:30.454562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 22:48:30.454562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:30.457509 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 385/774] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:30 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:30.464094 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472edf7-1eca-4bbf-95dc-f5843f912a93 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:30.500869 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c960e1a-d535-4b8b-b099-3e6d08d8e77d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:30.501922 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c960e1a-d535-4b8b-b099-3e6d08d8e77d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:30.502606 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c960e1a-d535-4b8b-b099-3e6d08d8e77d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:30.504246 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c960e1a-d535-4b8b-b099-3e6d08d8e77d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:30.505360 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c960e1a-d535-4b8b-b099-3e6d08d8e77d None admin] RBAC: Adding query filter params (interface=public, service_id=d10002765b6747a4bb2889160b99a572, region_id=RegionOne) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:30.506142 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c960e1a-d535-4b8b-b099-3e6d08d8e77d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:30.508571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c960e1a-d535-4b8b-b099-3e6d08d8e77d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:30.520065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 390/775] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jun 16 22:48:30 2022] GET /identity/v3/endpoints?service_id=d10002765b6747a4bb2889160b99a572&interface=public®ion_id=RegionOne => generated 194 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:30.526274 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c960e1a-d535-4b8b-b099-3e6d08d8e77d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:30.577428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:30.587695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:30.651607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2472edf7-1eca-4bbf-95dc-f5843f912a93 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:30.654745 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c960e1a-d535-4b8b-b099-3e6d08d8e77d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:30.693184 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a29123-916b-4bb2-a762-5c1ab06ab153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:30.694215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a29123-916b-4bb2-a762-5c1ab06ab153 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:30.694762 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a29123-916b-4bb2-a762-5c1ab06ab153 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:30.695265 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a29123-916b-4bb2-a762-5c1ab06ab153 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:30.696171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a29123-916b-4bb2-a762-5c1ab06ab153 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:30.697943 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a29123-916b-4bb2-a762-5c1ab06ab153 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:30.703849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 391/776] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jun 16 22:48:30 2022] GET /identity/v3/services => generated 772 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:30.744100 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f3df4-ac46-4dfa-961c-1de190fc49fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:30.744801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f3df4-ac46-4dfa-961c-1de190fc49fd None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:30.745088 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f3df4-ac46-4dfa-961c-1de190fc49fd None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:30.745365 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40f3df4-ac46-4dfa-961c-1de190fc49fd None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:30.746089 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f3df4-ac46-4dfa-961c-1de190fc49fd None admin] RBAC: Adding query filter params (name=network) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:30.746399 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f3df4-ac46-4dfa-961c-1de190fc49fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:30.747815 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40f3df4-ac46-4dfa-961c-1de190fc49fd None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:30.759509 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 386/777] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 22:48:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:30.765086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a29123-916b-4bb2-a762-5c1ab06ab153 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:30.875068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a29123-916b-4bb2-a762-5c1ab06ab153 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:30.886866 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:30.912872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ffd31-5201-4498-813e-e316ae8dbf9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:30.913563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ffd31-5201-4498-813e-e316ae8dbf9b None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:30.913852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ffd31-5201-4498-813e-e316ae8dbf9b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:30.914137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ffd31-5201-4498-813e-e316ae8dbf9b None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:30.914899 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ffd31-5201-4498-813e-e316ae8dbf9b None admin] RBAC: Adding query filter params (type=network) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:30.915215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ffd31-5201-4498-813e-e316ae8dbf9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:30.916632 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ffd31-5201-4498-813e-e316ae8dbf9b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:30.922787 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 392/778] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 22:48:30 2022] GET /identity/v3/services?type=network => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:30.928572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f3df4-ac46-4dfa-961c-1de190fc49fd None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:31.044956 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40f3df4-ac46-4dfa-961c-1de190fc49fd None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:31.069147 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104e3d1-49e1-4217-9cf5-fe2b7b3b6a08 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:31.069463 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104e3d1-49e1-4217-9cf5-fe2b7b3b6a08 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:31.069723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f104e3d1-49e1-4217-9cf5-fe2b7b3b6a08 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:31.071395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 393/779] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:31.076921 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc1ae4-76f5-4c4f-8aca-595cde94818e None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:31.077216 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc1ae4-76f5-4c4f-8aca-595cde94818e None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:31.077499 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bc1ae4-76f5-4c4f-8aca-595cde94818e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:31.086914 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14380f7-ac94-47e3-83d8-e68a44d1aed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:31.087585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14380f7-ac94-47e3-83d8-e68a44d1aed4 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:31.087875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14380f7-ac94-47e3-83d8-e68a44d1aed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:31.088151 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14380f7-ac94-47e3-83d8-e68a44d1aed4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:31.088952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14380f7-ac94-47e3-83d8-e68a44d1aed4 None admin] RBAC: Adding query filter params (interface=public, service_id=6a966ce4c8cf4928a8524c8f009ce802, region_id=RegionOne) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:31.089268 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14380f7-ac94-47e3-83d8-e68a44d1aed4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:31.090679 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14380f7-ac94-47e3-83d8-e68a44d1aed4 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:31.095524 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:31.096395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-24bc1ae4-76f5-4c4f-8aca-595cde94818e None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:31.100212 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 387/780] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jun 16 22:48:30 2022] GET /identity/v3/endpoints?service_id=6a966ce4c8cf4928a8524c8f009ce802&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:31.103106 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-24bc1ae4-76f5-4c4f-8aca-595cde94818e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:31.106061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14380f7-ac94-47e3-83d8-e68a44d1aed4 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:31.255701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 394/781] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:31.266207 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14380f7-ac94-47e3-83d8-e68a44d1aed4 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:31.294821 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc1ae4-76f5-4c4f-8aca-595cde94818e None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:31.308489 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff97be-490f-498f-a85c-19615a5691b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:31.309480 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff97be-490f-498f-a85c-19615a5691b3 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:31.309973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff97be-490f-498f-a85c-19615a5691b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:31.310548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ff97be-490f-498f-a85c-19615a5691b3 None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:31.311695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff97be-490f-498f-a85c-19615a5691b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:31.313290 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ff97be-490f-498f-a85c-19615a5691b3 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:31.323890 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 388/782] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jun 16 22:48:31 2022] GET /identity/v3/services => generated 772 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:31.452276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bc1ae4-76f5-4c4f-8aca-595cde94818e None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:31.454764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:31.505513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae070822-185f-4232-b92b-0e676b5ac4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:31.506331 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae070822-185f-4232-b92b-0e676b5ac4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:31.506622 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae070822-185f-4232-b92b-0e676b5ac4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:31.506908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae070822-185f-4232-b92b-0e676b5ac4b3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:31.527597 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae070822-185f-4232-b92b-0e676b5ac4b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:31.538744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-ae070822-185f-4232-b92b-0e676b5ac4b3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:48:31.538744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:31.538744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:31.538744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:31.538744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:31.538744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:31.538744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:31.538744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:31.538744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:31.538744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:31.538744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:31.538744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:31.538744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:31.538744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:31.538744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:31.538744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:31.539693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:31.539693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:31.539693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:31.539693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:31.539693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:31.539693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:31.539693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:31.539693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:31.539693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:31.539693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:31.539693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:31.539693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:31.539693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:31.539693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:31.539693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:31.539693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:31.540530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:31.540530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:31.540530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:31.540530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:31.540530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:31.540530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:31.540530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:31.540530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:31.540530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:31.540530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:31.540530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:48:31.540530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:31.540530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 395/783] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:48:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:31.547532 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff97be-490f-498f-a85c-19615a5691b3 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:31.622532 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:31.666562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ff97be-490f-498f-a85c-19615a5691b3 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:31.712828 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7e590-ba4e-401c-8fbf-92a9895c7124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:31.713877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7e590-ba4e-401c-8fbf-92a9895c7124 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:31.714373 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7e590-ba4e-401c-8fbf-92a9895c7124 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:31.714883 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad7e590-ba4e-401c-8fbf-92a9895c7124 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:31.715872 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7e590-ba4e-401c-8fbf-92a9895c7124 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:31.716387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7e590-ba4e-401c-8fbf-92a9895c7124 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:31.717991 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad7e590-ba4e-401c-8fbf-92a9895c7124 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:31.732513 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 389/784] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:48:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:31.737840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae070822-185f-4232-b92b-0e676b5ac4b3 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:31.869594 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae070822-185f-4232-b92b-0e676b5ac4b3 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:31.914156 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc663d-5fdd-499d-8b47-7d8be0300cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:31.914926 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc663d-5fdd-499d-8b47-7d8be0300cfa None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:31.915216 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc663d-5fdd-499d-8b47-7d8be0300cfa None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:31.915498 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc663d-5fdd-499d-8b47-7d8be0300cfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:31.934878 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc663d-5fdd-499d-8b47-7d8be0300cfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:31.936349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc663d-5fdd-499d-8b47-7d8be0300cfa None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:31.960092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-a2cc663d-5fdd-499d-8b47-7d8be0300cfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:31.960092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:31.960092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:31.960092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:31.960092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:31.960092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:31.960092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:31.960092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:31.960092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:31.960092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:31.960092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:31.960092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:31.960092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:31.960092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:31.960092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:31.963422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:31.963422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:31.963422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:31.963422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:31.963422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:31.963422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:31.963422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:31.963422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:31.963422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:31.963422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:31.963422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:31.963422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:31.963422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:31.963422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:31.963422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:31.963422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:31.963422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:31.964318 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:31.964318 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:31.964318 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:31.964318 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:31.964318 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 396/785] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:31.967290 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7e590-ba4e-401c-8fbf-92a9895c7124 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:31.987326 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769719b2-2132-4069-ab9f-22a73dfad112 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:31.987658 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769719b2-2132-4069-ab9f-22a73dfad112 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:31.987897 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769719b2-2132-4069-ab9f-22a73dfad112 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:31.990081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 397/786] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:48:31.995100 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded56ef-5d07-4d05-bdde-cfafd9dd4686 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:31.995390 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded56ef-5d07-4d05-bdde-cfafd9dd4686 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:31.995672 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded56ef-5d07-4d05-bdde-cfafd9dd4686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:32.027636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-3ded56ef-5d07-4d05-bdde-cfafd9dd4686 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:32.029094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-3ded56ef-5d07-4d05-bdde-cfafd9dd4686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:32.041079 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:32.136551 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad7e590-ba4e-401c-8fbf-92a9895c7124 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:32.156743 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 398/787] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:48:32.175188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:32.187588 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87472b0-4887-4ae9-8e2a-83558afdfc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:32.188273 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87472b0-4887-4ae9-8e2a-83558afdfc74 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:32.188562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87472b0-4887-4ae9-8e2a-83558afdfc74 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:32.188843 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87472b0-4887-4ae9-8e2a-83558afdfc74 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:32.189568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87472b0-4887-4ae9-8e2a-83558afdfc74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:32.189880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87472b0-4887-4ae9-8e2a-83558afdfc74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:32.191292 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87472b0-4887-4ae9-8e2a-83558afdfc74 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:32.205181 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 390/788] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:32.210088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded56ef-5d07-4d05-bdde-cfafd9dd4686 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:32.220455 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87472b0-4887-4ae9-8e2a-83558afdfc74 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:32.389413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87472b0-4887-4ae9-8e2a-83558afdfc74 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:32.397777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded56ef-5d07-4d05-bdde-cfafd9dd4686 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:32.426734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f9fa9-7cdf-4df5-bcbc-042bb9e6bef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:32.427468 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f9fa9-7cdf-4df5-bcbc-042bb9e6bef4 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:32.427757 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f9fa9-7cdf-4df5-bcbc-042bb9e6bef4 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:32.428061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f9fa9-7cdf-4df5-bcbc-042bb9e6bef4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:32.435502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f9fa9-7cdf-4df5-bcbc-042bb9e6bef4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:32.436307 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c6751-b2d1-4df2-bee7-aea908d6a1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:32.437042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f9fa9-7cdf-4df5-bcbc-042bb9e6bef4 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:32.437546 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c6751-b2d1-4df2-bee7-aea908d6a1d3 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:32.437999 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c6751-b2d1-4df2-bee7-aea908d6a1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:32.438424 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c6751-b2d1-4df2-bee7-aea908d6a1d3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:32.451091 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c6751-b2d1-4df2-bee7-aea908d6a1d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:32.452724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-148f9fa9-7cdf-4df5-bcbc-042bb9e6bef4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 22:48:32.452724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:32.452724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:32.452724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:32.452724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:32.452724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:32.452724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:32.452724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:32.452724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:32.452724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:32.452724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:32.452724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:32.452724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:32.452724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:32.452724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:32.452724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:32.453646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:32.453646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:32.453646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:32.453646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:32.453646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:32.453646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:32.453646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:32.453646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:32.453646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:32.453646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:32.453646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:32.453646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:32.453646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:32.453646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:32.453646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:32.453646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:32.453646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:32.453646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:32.454574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:32.454574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:32.454574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:32.454574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:32.454574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:32.454574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:32.454574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:32.454574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:32.454574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 22:48:32.454574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:32.454574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 391/789] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:48:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:32.456101 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba76cf1-4b19-43e6-ba04-3bd78e314c0d None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:32.456456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba76cf1-4b19-43e6-ba04-3bd78e314c0d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:32.456803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba76cf1-4b19-43e6-ba04-3bd78e314c0d None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:32.458438 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 392/790] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:32.459801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba76cf1-4b19-43e6-ba04-3bd78e314c0d None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:32.463314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-c38c6751-b2d1-4df2-bee7-aea908d6a1d3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:48:32.463314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:32.463314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:32.463314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:32.463314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:32.463314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:32.463314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:32.463314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:32.463314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:32.463314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:32.463314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:32.463314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:32.463314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:32.463314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:32.463314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:32.463314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:32.464286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:32.464286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:32.464286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:32.464286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:32.464286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:32.464286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:32.464286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:32.464286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:32.464286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:32.464286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:32.464286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:32.464286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:32.464286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:32.464286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:32.464286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:32.464286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:32.465110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:32.465110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:32.465110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:32.465110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:32.465110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:32.465110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:32.465110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:32.465110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:32.465110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:32.465110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:32.465110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:48:32.465110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:32.467747 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 399/791] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:48:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:32.469173 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb55ae3-71e7-421d-afdd-7c2b7e7e634a None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:32.469641 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb55ae3-71e7-421d-afdd-7c2b7e7e634a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:32.470088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb55ae3-71e7-421d-afdd-7c2b7e7e634a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:32.482768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:32.492887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-3cb55ae3-71e7-421d-afdd-7c2b7e7e634a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:32.494066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-3cb55ae3-71e7-421d-afdd-7c2b7e7e634a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:32.521557 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:32.578880 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 400/792] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:32.580133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb55ae3-71e7-421d-afdd-7c2b7e7e634a None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:32.586799 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba76cf1-4b19-43e6-ba04-3bd78e314c0d None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:32.627361 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fef627-a80e-425f-9b7d-9c4297e1f538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:32.628260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fef627-a80e-425f-9b7d-9c4297e1f538 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:32.628645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fef627-a80e-425f-9b7d-9c4297e1f538 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:32.629040 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fef627-a80e-425f-9b7d-9c4297e1f538 None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:32.630044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fef627-a80e-425f-9b7d-9c4297e1f538 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:32.630554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fef627-a80e-425f-9b7d-9c4297e1f538 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:32.632248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0fef627-a80e-425f-9b7d-9c4297e1f538 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:32.651947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 393/793] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jun 16 22:48:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:32.653247 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fef627-a80e-425f-9b7d-9c4297e1f538 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:32.698369 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb55ae3-71e7-421d-afdd-7c2b7e7e634a None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:32.735401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ff843-4e4e-46bd-9bba-a95c85efff7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:32.736379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ff843-4e4e-46bd-9bba-a95c85efff7b None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:32.736825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ff843-4e4e-46bd-9bba-a95c85efff7b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:32.737273 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e31ff843-4e4e-46bd-9bba-a95c85efff7b None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:32.738187 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ff843-4e4e-46bd-9bba-a95c85efff7b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:32.739846 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ff843-4e4e-46bd-9bba-a95c85efff7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:32.741451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e31ff843-4e4e-46bd-9bba-a95c85efff7b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:32.744106 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:32.756622 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 401/794] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:48:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:32.756622 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ff843-4e4e-46bd-9bba-a95c85efff7b None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:32.769519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fef627-a80e-425f-9b7d-9c4297e1f538 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:32.803095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:32.809327 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a83b0e-2fea-41c5-81ef-25ae20d3ba99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:32.810899 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a83b0e-2fea-41c5-81ef-25ae20d3ba99 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:32.811291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a83b0e-2fea-41c5-81ef-25ae20d3ba99 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:32.811738 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a83b0e-2fea-41c5-81ef-25ae20d3ba99 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:32.827725 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a83b0e-2fea-41c5-81ef-25ae20d3ba99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:32.837441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-d1a83b0e-2fea-41c5-81ef-25ae20d3ba99 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:32.837441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:32.837441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:32.837441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:32.837441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:32.837441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:32.837441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:32.837441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:32.837441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:32.837441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:32.837441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:32.837441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:32.837441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:32.837441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:32.837441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:32.837441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:32.838396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:32.838396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:32.838396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:32.838396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:32.838396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:32.838396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:32.838396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:32.838396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:32.838396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:32.838396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:32.838396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:32.838396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:32.838396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:32.838396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:32.838396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:32.838396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:32.839281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:32.839281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:32.839281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:32.839281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:32.839281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:32.839281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:32.839281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:32.839281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:32.839281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:32.839281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:32.839281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:32.839281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:32.842678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 394/795] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:32.843749 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a83b0e-2fea-41c5-81ef-25ae20d3ba99 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:32.878039 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e31ff843-4e4e-46bd-9bba-a95c85efff7b None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:32.924968 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62742b-4d8e-415d-9aee-12245f44dd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:32.926203 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62742b-4d8e-415d-9aee-12245f44dd66 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:32.926535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62742b-4d8e-415d-9aee-12245f44dd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:32.927243 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb62742b-4d8e-415d-9aee-12245f44dd66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:32.943570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62742b-4d8e-415d-9aee-12245f44dd66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:32.945554 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb62742b-4d8e-415d-9aee-12245f44dd66 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:32.969642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-fb62742b-4d8e-415d-9aee-12245f44dd66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:32.969642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:32.969642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:32.969642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:32.969642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:32.969642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:32.969642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:32.969642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:32.969642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:32.969642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:32.969642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:32.969642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:32.969642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:32.969642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:32.969642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:32.970514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:32.970514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:32.970514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:32.970514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:32.970514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:32.970514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:32.970514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:32.970514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:32.970514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:32.970514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:32.970514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:32.970514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:32.970514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:32.970514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:32.970514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:32.970514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:32.970514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:32.971423 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:32.971423 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:32.971423 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:32.971423 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:32.971638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a83b0e-2fea-41c5-81ef-25ae20d3ba99 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:32.975069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 402/796] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:32.976094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62742b-4d8e-415d-9aee-12245f44dd66 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:33.010620 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5f833-8e67-465d-a589-2337d786acd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:33.011622 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5f833-8e67-465d-a589-2337d786acd4 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:33.012061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5f833-8e67-465d-a589-2337d786acd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:33.012463 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5f833-8e67-465d-a589-2337d786acd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:33.024450 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5f833-8e67-465d-a589-2337d786acd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:33.026025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc5f833-8e67-465d-a589-2337d786acd4 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:33.041645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:33.045733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-1bc5f833-8e67-465d-a589-2337d786acd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:33.045733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:33.045733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:33.045733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:33.045733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:33.045733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:33.045733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:33.045733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:33.045733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:33.045733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:33.045733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:33.045733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:33.045733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:33.045733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:33.045733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:33.047577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:33.047577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:33.047577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:33.047577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:33.047577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:33.047577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:33.047577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:33.047577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:33.047577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:33.047577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:33.047577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:33.047577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:33.047577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:33.047577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:33.047577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:33.047577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:33.047577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:33.048607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:33.048607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:33.048607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:33.048607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:33.049137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 395/797] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:33.050319 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5f833-8e67-465d-a589-2337d786acd4 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:33.073902 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:33.102690 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb62742b-4d8e-415d-9aee-12245f44dd66 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:33.144925 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966c96b-c2d3-45a8-af6d-600df5700aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:33.145920 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966c96b-c2d3-45a8-af6d-600df5700aae None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:33.146354 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966c96b-c2d3-45a8-af6d-600df5700aae None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:33.146797 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966c96b-c2d3-45a8-af6d-600df5700aae None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:33.147686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966c96b-c2d3-45a8-af6d-600df5700aae None admin] RBAC: Adding query filter params (name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:33.148139 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966c96b-c2d3-45a8-af6d-600df5700aae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:33.149687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966c96b-c2d3-45a8-af6d-600df5700aae None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:33.159796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 403/798] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:48:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:33.166228 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966c96b-c2d3-45a8-af6d-600df5700aae None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:33.172341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc5f833-8e67-465d-a589-2337d786acd4 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:33.218398 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc547d11-4f68-4494-83a3-8eab63ed0572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:33.219299 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc547d11-4f68-4494-83a3-8eab63ed0572 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:33.219713 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc547d11-4f68-4494-83a3-8eab63ed0572 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:33.220027 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc547d11-4f68-4494-83a3-8eab63ed0572 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:33.220966 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc547d11-4f68-4494-83a3-8eab63ed0572 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:33.221349 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc547d11-4f68-4494-83a3-8eab63ed0572 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:33.222876 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc547d11-4f68-4494-83a3-8eab63ed0572 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:33.239656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 396/799] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:33.243946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc547d11-4f68-4494-83a3-8eab63ed0572 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:33.288385 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966c96b-c2d3-45a8-af6d-600df5700aae None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:33.302574 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:33.323788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:33.326559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e3a2be-be74-4f8d-84be-ea161652a2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:33.327277 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3a2be-be74-4f8d-84be-ea161652a2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:33.327545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3a2be-be74-4f8d-84be-ea161652a2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:33.327842 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e3a2be-be74-4f8d-84be-ea161652a2f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:33.328562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e3a2be-be74-4f8d-84be-ea161652a2f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:33.328898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e3a2be-be74-4f8d-84be-ea161652a2f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:33.330220 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e3a2be-be74-4f8d-84be-ea161652a2f4 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:33.342973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 404/800] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:33.344145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdf0b5-7c8c-4312-a901-195772dc0c47 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:33.344433 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdf0b5-7c8c-4312-a901-195772dc0c47 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:33.344716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdf0b5-7c8c-4312-a901-195772dc0c47 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:33.346077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 405/801] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:33.346640 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cdf0b5-7c8c-4312-a901-195772dc0c47 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:33.358227 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc547d11-4f68-4494-83a3-8eab63ed0572 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:33.393564 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0db2c-4cb0-4516-ac35-b6733a6d3479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:33.394365 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0db2c-4cb0-4516-ac35-b6733a6d3479 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:33.394674 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0db2c-4cb0-4516-ac35-b6733a6d3479 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:33.394956 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa0db2c-4cb0-4516-ac35-b6733a6d3479 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:33.406121 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0db2c-4cb0-4516-ac35-b6733a6d3479 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:33.407622 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa0db2c-4cb0-4516-ac35-b6733a6d3479 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:33.425833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-8aa0db2c-4cb0-4516-ac35-b6733a6d3479 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:33.425833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:33.425833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:33.425833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:33.425833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:33.425833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:33.425833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:33.425833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:33.425833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:33.425833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:33.425833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:33.425833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:33.425833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:33.425833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:33.425833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:33.427326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:33.427326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:33.427326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:33.427326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:33.427326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:33.427326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:33.427326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:33.427326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:33.427326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:33.427326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:33.427326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:33.427326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:33.427326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:33.427326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:33.427326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:33.427326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:33.427326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:33.433355 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:33.433355 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:33.433355 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:33.433355 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:33.433355 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 397/802] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:33.433355 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0db2c-4cb0-4516-ac35-b6733a6d3479 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:33.461089 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cdf0b5-7c8c-4312-a901-195772dc0c47 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:33.498635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87483d-4d83-40c8-85a0-a55cbcf8a901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:33.499368 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87483d-4d83-40c8-85a0-a55cbcf8a901 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:33.499659 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87483d-4d83-40c8-85a0-a55cbcf8a901 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:33.499941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87483d-4d83-40c8-85a0-a55cbcf8a901 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:33.506866 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87483d-4d83-40c8-85a0-a55cbcf8a901 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:33.508297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87483d-4d83-40c8-85a0-a55cbcf8a901 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:33.523287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-ea87483d-4d83-40c8-85a0-a55cbcf8a901 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:33.523287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:33.523287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:33.523287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:33.523287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:33.523287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:33.523287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:33.523287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:33.523287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:33.523287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:33.523287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:33.523287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:33.523287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:33.523287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:33.523287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:33.524214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:33.524214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:33.524214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:33.524214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:33.524214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:33.524214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:33.524214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:33.524214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:33.524214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:33.524214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:33.524214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:33.524214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:33.524214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:33.524214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:33.524214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:33.524214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:33.524214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:33.525077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:33.525077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:33.525077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:33.525077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:33.525077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:33.525077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:33.525077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:33.525077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:33.526759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 406/803] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:33.528744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65022b77-9997-46d4-a9f8-2a5126823b31 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:33.529467 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65022b77-9997-46d4-a9f8-2a5126823b31 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:33.530299 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65022b77-9997-46d4-a9f8-2a5126823b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:33.539173 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa0db2c-4cb0-4516-ac35-b6733a6d3479 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:33.549422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-65022b77-9997-46d4-a9f8-2a5126823b31 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:33.550968 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-65022b77-9997-46d4-a9f8-2a5126823b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:33.568999 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:33.578008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67697662-e763-41fd-99f3-1d728e49254e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:33.578290 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:33.579856 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67697662-e763-41fd-99f3-1d728e49254e None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:33.580144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67697662-e763-41fd-99f3-1d728e49254e None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:33.580427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67697662-e763-41fd-99f3-1d728e49254e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:33.588951 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67697662-e763-41fd-99f3-1d728e49254e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:33.590395 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67697662-e763-41fd-99f3-1d728e49254e None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:33.605818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-67697662-e763-41fd-99f3-1d728e49254e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 22:48:33.605818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:33.605818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:33.605818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:33.605818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:33.605818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:33.605818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:33.605818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:33.605818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:33.605818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:33.605818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:33.605818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:33.605818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:33.605818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:33.605818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:33.605818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:33.606797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:33.606797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:33.606797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:33.606797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:33.606797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:33.606797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:33.606797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:33.606797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:33.606797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:33.606797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:33.606797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:33.606797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:33.606797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:33.606797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:33.606797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:33.606797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:33.606797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:33.606797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:33.607704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:33.607704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:33.607704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:33.607704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:33.607704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:33.607704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:33.607704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:33.607704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:33.607704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 22:48:33.607704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:33.607704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 398/804] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jun 16 22:48:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:33.608303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67697662-e763-41fd-99f3-1d728e49254e None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:33.625653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 407/805] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:33.626352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65022b77-9997-46d4-a9f8-2a5126823b31 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:33.731777 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67697662-e763-41fd-99f3-1d728e49254e None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:33.736522 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65022b77-9997-46d4-a9f8-2a5126823b31 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:33.769827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98669b35-7334-402f-b824-c2060a376740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:33.770535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98669b35-7334-402f-b824-c2060a376740 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:33.770915 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98669b35-7334-402f-b824-c2060a376740 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:33.771260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98669b35-7334-402f-b824-c2060a376740 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:33.772049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98669b35-7334-402f-b824-c2060a376740 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:33.772424 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98669b35-7334-402f-b824-c2060a376740 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:33.773224 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc10a6a-a817-4f0e-99bc-36a28075e3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:33.774107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc10a6a-a817-4f0e-99bc-36a28075e3fb None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:33.774209 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98669b35-7334-402f-b824-c2060a376740 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:33.774399 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc10a6a-a817-4f0e-99bc-36a28075e3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:33.774704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc10a6a-a817-4f0e-99bc-36a28075e3fb None admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:33.775497 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc10a6a-a817-4f0e-99bc-36a28075e3fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:33.775813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc10a6a-a817-4f0e-99bc-36a28075e3fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:33.777239 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc10a6a-a817-4f0e-99bc-36a28075e3fb None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:33.787940 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 399/806] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:33.788738 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98669b35-7334-402f-b824-c2060a376740 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:33.791072 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 408/807] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:33.791072 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc10a6a-a817-4f0e-99bc-36a28075e3fb None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:33.815760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:33.838093 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:33.886924 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98669b35-7334-402f-b824-c2060a376740 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:33.890767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc10a6a-a817-4f0e-99bc-36a28075e3fb None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:33.920116 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40338bf0-4bfe-4909-b116-8cc697f115ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:33.920786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40338bf0-4bfe-4909-b116-8cc697f115ed None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:33.921073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40338bf0-4bfe-4909-b116-8cc697f115ed None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:33.921422 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40338bf0-4bfe-4909-b116-8cc697f115ed None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:33.922183 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40338bf0-4bfe-4909-b116-8cc697f115ed None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:33.922494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40338bf0-4bfe-4909-b116-8cc697f115ed None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:33.923936 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40338bf0-4bfe-4909-b116-8cc697f115ed None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:33.924292 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0906b-6f8a-46f8-abc2-3d895846f5e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:33.925095 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0906b-6f8a-46f8-abc2-3d895846f5e3 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:33.925383 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0906b-6f8a-46f8-abc2-3d895846f5e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:33.925663 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f0906b-6f8a-46f8-abc2-3d895846f5e3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:33.926316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0906b-6f8a-46f8-abc2-3d895846f5e3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:33.927688 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f0906b-6f8a-46f8-abc2-3d895846f5e3 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:33.939557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-48f0906b-6f8a-46f8-abc2-3d895846f5e3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 22:48:33.939557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:33.939557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:33.939557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:33.939557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:33.939557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:33.939557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:33.939557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:33.939557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 22:48:33.939557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:33.939557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:33.939557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:33.939557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:33.939557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:33.939557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:33.939557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:33.945923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:33.945923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:33.945923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:33.945923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:33.945923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:33.945923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:33.945923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:33.945923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:33.945923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:33.945923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:33.945923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:33.945923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:33.945923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:33.945923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:33.945923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:33.945923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:33.946971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:33.946971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:33.946971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:33.946971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:33.946971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:33.946971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:33.946971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:33.946971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:33.946971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:33.946971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:33.946971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:33.946971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:33.946971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:33.946971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 22:48:33.946971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:33.948012 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 400/808] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jun 16 22:48:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:33.948094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 409/809] 10.222.0.17 () {56 vars in 1129 bytes} [Thu Jun 16 22:48:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:33.948094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0906b-6f8a-46f8-abc2-3d895846f5e3 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:33.948253 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b12a9f-4c35-4bd7-b657-1c661bae0d22 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:33.948253 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b12a9f-4c35-4bd7-b657-1c661bae0d22 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:33.948253 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b12a9f-4c35-4bd7-b657-1c661bae0d22 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:33.948253 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 401/810] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:33.948253 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b12a9f-4c35-4bd7-b657-1c661bae0d22 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:34.029717 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:34.045154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f0906b-6f8a-46f8-abc2-3d895846f5e3 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:34.047010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b12a9f-4c35-4bd7-b657-1c661bae0d22 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:34.054617 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:34.078196 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19830e38-d91b-416a-b152-07bf30b0e812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:34.079042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19830e38-d91b-416a-b152-07bf30b0e812 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:34.079334 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19830e38-d91b-416a-b152-07bf30b0e812 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:34.079614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19830e38-d91b-416a-b152-07bf30b0e812 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:34.081345 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f1d43-dc6f-4a54-9d59-7c5a7dbd588b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:34.082220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f1d43-dc6f-4a54-9d59-7c5a7dbd588b None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:34.082525 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f1d43-dc6f-4a54-9d59-7c5a7dbd588b None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:34.082841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211f1d43-dc6f-4a54-9d59-7c5a7dbd588b None admin] PATH_INFO: `/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/bef518c9430e4274b1066241f821f4a1/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:34.084786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f1d43-dc6f-4a54-9d59-7c5a7dbd588b None admin] RBAC: Authorizing `identity:create_grant(project_id=bc48914270df4db996bd2a4b5b9fc777, user_id=bef518c9430e4274b1066241f821f4a1, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:34.086200 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211f1d43-dc6f-4a54-9d59-7c5a7dbd588b None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:34.087251 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19830e38-d91b-416a-b152-07bf30b0e812 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:34.088701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19830e38-d91b-416a-b152-07bf30b0e812 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:34.098709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 402/811] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:48:33 2022] PUT /identity/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/bef518c9430e4274b1066241f821f4a1/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:48:34.099497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f1d43-dc6f-4a54-9d59-7c5a7dbd588b None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:34.103708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-19830e38-d91b-416a-b152-07bf30b0e812 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 22:48:34.103708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:34.103708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:34.103708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:34.103708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:34.103708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:34.103708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:34.103708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:34.103708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:34.103708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:34.103708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:34.103708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:34.103708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:34.103708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:34.103708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:34.103708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:34.104868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:34.104868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:34.104868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:34.104868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:34.104868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:34.104868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:34.104868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:34.104868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:34.104868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:34.104868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:34.104868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:34.104868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:34.104868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:34.104868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:34.104868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:34.104868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:34.104868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:34.104868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:34.106051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:34.106051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:34.106051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:34.106051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:34.106051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:34.106051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:34.106051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:34.106051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:34.106051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 22:48:34.106051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:34.110068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 410/812] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 22:48:33 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:34.111195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19830e38-d91b-416a-b152-07bf30b0e812 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:34.225405 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211f1d43-dc6f-4a54-9d59-7c5a7dbd588b None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:34.241846 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19830e38-d91b-416a-b152-07bf30b0e812 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:34.272584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023de3a-53ed-48f0-9576-4d6ba1f99e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:34.273488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023de3a-53ed-48f0-9576-4d6ba1f99e49 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:34.273975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023de3a-53ed-48f0-9576-4d6ba1f99e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:34.274263 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b023de3a-53ed-48f0-9576-4d6ba1f99e49 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:34.290054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec28c08-d317-45aa-ab45-049e2051033e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:34.290748 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec28c08-d317-45aa-ab45-049e2051033e None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:34.291042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec28c08-d317-45aa-ab45-049e2051033e None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:34.291324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec28c08-d317-45aa-ab45-049e2051033e None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:34.292034 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec28c08-d317-45aa-ab45-049e2051033e None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:34.292344 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec28c08-d317-45aa-ab45-049e2051033e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:34.293710 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec28c08-d317-45aa-ab45-049e2051033e None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:34.298472 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 411/813] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jun 16 22:48:34 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:34.299832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372f081-2ef7-400b-8cca-71c5de43b784 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:34.300131 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372f081-2ef7-400b-8cca-71c5de43b784 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:34.300414 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d372f081-2ef7-400b-8cca-71c5de43b784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:34.314350 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:34.318960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-d372f081-2ef7-400b-8cca-71c5de43b784 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:34.320135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-d372f081-2ef7-400b-8cca-71c5de43b784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:34.324258 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:34.351664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023de3a-53ed-48f0-9576-4d6ba1f99e49 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:34.353122 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b023de3a-53ed-48f0-9576-4d6ba1f99e49 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:34.393466 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 412/814] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:34.394120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372f081-2ef7-400b-8cca-71c5de43b784 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:34.463588 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 403/815] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:48:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:48:34.464433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023de3a-53ed-48f0-9576-4d6ba1f99e49 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:34.505627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d372f081-2ef7-400b-8cca-71c5de43b784 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:34.533365 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:34.542875 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abdbdb-6331-460c-85ab-fcb3533bee54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:34.543484 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abdbdb-6331-460c-85ab-fcb3533bee54 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:34.543767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abdbdb-6331-460c-85ab-fcb3533bee54 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:34.544045 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0abdbdb-6331-460c-85ab-fcb3533bee54 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:34.544876 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0abdbdb-6331-460c-85ab-fcb3533bee54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:34.545186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0abdbdb-6331-460c-85ab-fcb3533bee54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:34.546653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0abdbdb-6331-460c-85ab-fcb3533bee54 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:34.555492 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:34.565611 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 413/816] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jun 16 22:48:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:34.566415 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abdbdb-6331-460c-85ab-fcb3533bee54 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:34.575941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b023de3a-53ed-48f0-9576-4d6ba1f99e49 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:34.610724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9f03e-2db4-4eee-a017-9b500242abed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:34.611392 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9f03e-2db4-4eee-a017-9b500242abed None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:34.611678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9f03e-2db4-4eee-a017-9b500242abed None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:34.611956 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea9f03e-2db4-4eee-a017-9b500242abed None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:34.612657 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9f03e-2db4-4eee-a017-9b500242abed None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:34.612961 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9f03e-2db4-4eee-a017-9b500242abed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:34.614449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea9f03e-2db4-4eee-a017-9b500242abed None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:34.619156 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 404/817] 10.222.0.17 () {56 vars in 1124 bytes} [Thu Jun 16 22:48:34 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:34.619921 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9f03e-2db4-4eee-a017-9b500242abed None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:34.679658 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0abdbdb-6331-460c-85ab-fcb3533bee54 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:34.716793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c57929-6359-4483-b532-6c39352655a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:34.717584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c57929-6359-4483-b532-6c39352655a0 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:34.717869 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c57929-6359-4483-b532-6c39352655a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:34.718145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c57929-6359-4483-b532-6c39352655a0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:34.718820 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c57929-6359-4483-b532-6c39352655a0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:34.720249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c57929-6359-4483-b532-6c39352655a0 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:34.724837 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea9f03e-2db4-4eee-a017-9b500242abed None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:34.732394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-d5c57929-6359-4483-b532-6c39352655a0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 22:48:34.732394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:34.732394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:34.732394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:34.732394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:34.732394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:34.732394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:34.732394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:34.732394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 22:48:34.732394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:34.732394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:34.732394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:34.732394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:34.732394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:34.732394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:34.732394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:34.732394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:34.733379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:34.733379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:34.733379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:34.733379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:34.733379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:34.733379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:34.733379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:34.733379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:34.733379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:34.733379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:34.733379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:34.733379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:34.733379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:34.733379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:34.733379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:34.733379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:34.739427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:34.739427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:34.739427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:34.739427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:34.739427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:34.739427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:34.739427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:34.739427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:34.739427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:34.739427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:34.739427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:34.739427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:34.739427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 22:48:34.739427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:34.739427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 414/818] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:34 2022] GET /identity/v3/services/network => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:34.740393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c57929-6359-4483-b532-6c39352655a0 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:34.761121 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:34.767950 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1ce8c-bb8c-4dc3-9368-5e26e1983c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:34.768850 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1ce8c-bb8c-4dc3-9368-5e26e1983c67 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:34.769206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1ce8c-bb8c-4dc3-9368-5e26e1983c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:34.769554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c1ce8c-bb8c-4dc3-9368-5e26e1983c67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:34.780839 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1ce8c-bb8c-4dc3-9368-5e26e1983c67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:34.782357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c1ce8c-bb8c-4dc3-9368-5e26e1983c67 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:34.800487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-44c1ce8c-bb8c-4dc3-9368-5e26e1983c67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:34.800487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:34.800487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:34.800487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:34.800487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:34.800487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:34.800487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:34.800487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:34.800487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:34.800487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:34.800487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:34.800487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:34.800487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:34.800487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:34.800487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:34.801695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:34.801695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:34.801695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:34.801695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:34.801695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:34.801695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:34.801695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:34.801695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:34.801695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:34.801695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:34.801695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:34.801695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:34.801695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:34.801695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:34.801695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:34.801695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:34.801695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:34.802941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:34.802941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:34.802941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:34.802941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:34.804815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:34.806863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 405/819] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:34.807697 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1ce8c-bb8c-4dc3-9368-5e26e1983c67 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:34.846819 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c57929-6359-4483-b532-6c39352655a0 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:34.882921 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3afd8b2-e94e-490b-b60c-ccf8e7505ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:34.883720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afd8b2-e94e-490b-b60c-ccf8e7505ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:34.884001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afd8b2-e94e-490b-b60c-ccf8e7505ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:34.884277 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3afd8b2-e94e-490b-b60c-ccf8e7505ef2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:34.895896 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3afd8b2-e94e-490b-b60c-ccf8e7505ef2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:34.897330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3afd8b2-e94e-490b-b60c-ccf8e7505ef2 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:34.915301 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c1ce8c-bb8c-4dc3-9368-5e26e1983c67 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:34.916528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-c3afd8b2-e94e-490b-b60c-ccf8e7505ef2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:34.916528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:34.916528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:34.916528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:34.916528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:34.916528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:34.916528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:34.916528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:34.916528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:34.916528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:34.916528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:34.916528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:34.916528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:34.916528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:34.916528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:34.917562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:34.917562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:34.917562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:34.917562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:34.917562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:34.917562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:34.917562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:34.917562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:34.917562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:34.917562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:34.917562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:34.917562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:34.917562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:34.917562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:34.917562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:34.917562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:34.917562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:34.923443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:34.923443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:34.923443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:34.923443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:34.923443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 415/820] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:34.923443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3afd8b2-e94e-490b-b60c-ccf8e7505ef2 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:34.957906 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2f11b-868a-4c41-afe2-f33cc4315ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:34.958688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2f11b-868a-4c41-afe2-f33cc4315ab5 None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:34.959044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2f11b-868a-4c41-afe2-f33cc4315ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:34.959498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2f11b-868a-4c41-afe2-f33cc4315ab5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:34.960448 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2f11b-868a-4c41-afe2-f33cc4315ab5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:34.962146 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2f11b-868a-4c41-afe2-f33cc4315ab5 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:34.984899 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 406/821] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jun 16 22:48:34 2022] POST /identity/v3/endpoints => generated 346 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:48:34.985599 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2f11b-868a-4c41-afe2-f33cc4315ab5 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:35.037022 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:35.062490 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3afd8b2-e94e-490b-b60c-ccf8e7505ef2 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:35.079111 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:35.107053 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719742d6-32c8-4971-8637-449a4d21901d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:35.107932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719742d6-32c8-4971-8637-449a4d21901d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:35.108217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719742d6-32c8-4971-8637-449a4d21901d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:35.108571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719742d6-32c8-4971-8637-449a4d21901d None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:35.109386 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719742d6-32c8-4971-8637-449a4d21901d None admin] RBAC: Adding query filter params (name=network) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:35.109769 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719742d6-32c8-4971-8637-449a4d21901d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:35.111496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719742d6-32c8-4971-8637-449a4d21901d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:35.115848 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2f11b-868a-4c41-afe2-f33cc4315ab5 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:35.122224 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 416/822] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 22:48:34 2022] GET /identity/v3/services?name=network => generated 124 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:35.123486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719742d6-32c8-4971-8637-449a4d21901d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:35.157336 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31734b34-1133-412e-9571-aa6168b07488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:35.158101 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31734b34-1133-412e-9571-aa6168b07488 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:35.158486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31734b34-1133-412e-9571-aa6168b07488 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:35.158911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31734b34-1133-412e-9571-aa6168b07488 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:35.159730 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31734b34-1133-412e-9571-aa6168b07488 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:35.161032 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31734b34-1133-412e-9571-aa6168b07488 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:35.162492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31734b34-1133-412e-9571-aa6168b07488 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:35.175183 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 407/823] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:35.176684 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31734b34-1133-412e-9571-aa6168b07488 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:35.257220 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719742d6-32c8-4971-8637-449a4d21901d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:35.293740 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:35.295536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690e8c6-499b-4470-ab6a-709f4ecd86ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:35.296218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690e8c6-499b-4470-ab6a-709f4ecd86ca None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:35.296505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690e8c6-499b-4470-ab6a-709f4ecd86ca None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:35.296784 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5690e8c6-499b-4470-ab6a-709f4ecd86ca None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:35.297581 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5690e8c6-499b-4470-ab6a-709f4ecd86ca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:35.297861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31734b34-1133-412e-9571-aa6168b07488 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:35.297968 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5690e8c6-499b-4470-ab6a-709f4ecd86ca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:35.299369 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5690e8c6-499b-4470-ab6a-709f4ecd86ca None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:35.311956 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 417/824] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:35.317717 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690e8c6-499b-4470-ab6a-709f4ecd86ca None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:35.335777 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18782774-8717-4943-a0be-2c64d381eecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:35.336458 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18782774-8717-4943-a0be-2c64d381eecb None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:35.336840 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18782774-8717-4943-a0be-2c64d381eecb None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:35.337127 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18782774-8717-4943-a0be-2c64d381eecb None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:35.337926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18782774-8717-4943-a0be-2c64d381eecb None admin] RBAC: Adding query filter params (type=network) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:35.338241 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18782774-8717-4943-a0be-2c64d381eecb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:35.339656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18782774-8717-4943-a0be-2c64d381eecb None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:35.345417 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 408/825] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 22:48:35 2022] GET /identity/v3/services?type=network => generated 350 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:35.346223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18782774-8717-4943-a0be-2c64d381eecb None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:35.360437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:35.435514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5690e8c6-499b-4470-ab6a-709f4ecd86ca None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:35.452833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18782774-8717-4943-a0be-2c64d381eecb None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:35.474894 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfcc681-808a-41e4-b0d4-a2b152922581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:35.476015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfcc681-808a-41e4-b0d4-a2b152922581 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:35.476446 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfcc681-808a-41e4-b0d4-a2b152922581 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:35.477006 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfcc681-808a-41e4-b0d4-a2b152922581 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:35.485284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfcc681-808a-41e4-b0d4-a2b152922581 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:35.487363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfcc681-808a-41e4-b0d4-a2b152922581 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:35.488197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70497b-67b3-4e32-bb0f-c1782ee9c3ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:35.489178 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70497b-67b3-4e32-bb0f-c1782ee9c3ee None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:35.489466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70497b-67b3-4e32-bb0f-c1782ee9c3ee None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:35.489741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff70497b-67b3-4e32-bb0f-c1782ee9c3ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:35.496560 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70497b-67b3-4e32-bb0f-c1782ee9c3ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:35.497997 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff70497b-67b3-4e32-bb0f-c1782ee9c3ee None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:35.503687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-ebfcc681-808a-41e4-b0d4-a2b152922581 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:35.503687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:35.503687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:35.503687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:35.503687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:35.503687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:35.503687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:35.503687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:35.503687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:35.503687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:35.503687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:35.503687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:35.503687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:35.503687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:35.503687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:35.505275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:35.505275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:35.505275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:35.505275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:35.505275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:35.505275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:35.505275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:35.505275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:35.505275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:35.505275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:35.505275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:35.505275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:35.505275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:35.505275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:35.505275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:35.505275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:35.505275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:35.506247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:35.506247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:35.506247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:35.506247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:35.506247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:35.506247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:35.506247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:35.506247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:35.508327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 418/826] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:35.509216 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfcc681-808a-41e4-b0d4-a2b152922581 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:35.516699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-ff70497b-67b3-4e32-bb0f-c1782ee9c3ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:35.516699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:35.516699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:35.516699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:35.516699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:35.516699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:35.516699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:35.516699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:35.516699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:35.516699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:35.516699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:35.516699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:35.516699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:35.516699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:35.516699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:35.517774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:35.517774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:35.517774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:35.517774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:35.517774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:35.517774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:35.517774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:35.517774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:35.517774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:35.517774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:35.517774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:35.517774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:35.517774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:35.517774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:35.517774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:35.517774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:35.517774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:35.518861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:35.518861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:35.518861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:35.518861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:35.518861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:35.518861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:35.518861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:35.518861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:35.520455 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 409/827] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:35.521499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70497b-67b3-4e32-bb0f-c1782ee9c3ee None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:35.547522 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:35.622535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:35.623549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff70497b-67b3-4e32-bb0f-c1782ee9c3ee None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:35.630898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfcc681-808a-41e4-b0d4-a2b152922581 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:35.664852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f1dad-d668-4784-951b-599e2f0dd699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:35.665622 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f1dad-d668-4784-951b-599e2f0dd699 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:35.665912 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f1dad-d668-4784-951b-599e2f0dd699 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:35.666192 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229f1dad-d668-4784-951b-599e2f0dd699 None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:35.667000 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f1dad-d668-4784-951b-599e2f0dd699 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:35.667309 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f1dad-d668-4784-951b-599e2f0dd699 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:35.668731 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-229f1dad-d668-4784-951b-599e2f0dd699 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:35.671332 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218bf54f-a8d7-4c8a-a32a-2f6d4c7405b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:35.672153 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218bf54f-a8d7-4c8a-a32a-2f6d4c7405b1 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:35.672583 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218bf54f-a8d7-4c8a-a32a-2f6d4c7405b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:35.673004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218bf54f-a8d7-4c8a-a32a-2f6d4c7405b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:35.673879 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218bf54f-a8d7-4c8a-a32a-2f6d4c7405b1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:35.675421 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218bf54f-a8d7-4c8a-a32a-2f6d4c7405b1 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:35.686530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 410/828] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:35.687537 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f1dad-d668-4784-951b-599e2f0dd699 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:35.707119 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 419/829] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jun 16 22:48:35 2022] POST /identity/v3/endpoints => generated 327 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:48:35.709088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218bf54f-a8d7-4c8a-a32a-2f6d4c7405b1 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:35.779024 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:35.798454 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229f1dad-d668-4784-951b-599e2f0dd699 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:35.855418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddd377-1f39-47f0-ac48-c1141f1bf4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:35.856114 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddd377-1f39-47f0-ac48-c1141f1bf4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:35.856398 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddd377-1f39-47f0-ac48-c1141f1bf4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:35.856671 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddd377-1f39-47f0-ac48-c1141f1bf4c4 None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:35.857473 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddd377-1f39-47f0-ac48-c1141f1bf4c4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:35.857788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddd377-1f39-47f0-ac48-c1141f1bf4c4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:35.861499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ddd377-1f39-47f0-ac48-c1141f1bf4c4 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:35.886652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 411/830] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:35.893932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddd377-1f39-47f0-ac48-c1141f1bf4c4 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:35.908357 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218bf54f-a8d7-4c8a-a32a-2f6d4c7405b1 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:35.969866 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c671d0-41a5-4d40-97be-32d26ba33efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:35.971242 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c671d0-41a5-4d40-97be-32d26ba33efc None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:35.971711 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c671d0-41a5-4d40-97be-32d26ba33efc None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:35.972354 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c671d0-41a5-4d40-97be-32d26ba33efc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:35.973822 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c671d0-41a5-4d40-97be-32d26ba33efc None admin] RBAC: Adding query filter params (role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=4180f2593149437fa27effd8d1338d10) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:35.974557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c671d0-41a5-4d40-97be-32d26ba33efc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:35.976323 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c671d0-41a5-4d40-97be-32d26ba33efc None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:35.984673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 420/831] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:48:35 2022] GET /identity/v3/role_assignments?user.id=4180f2593149437fa27effd8d1338d10&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 608 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:36.038123 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ddd377-1f39-47f0-ac48-c1141f1bf4c4 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:36.076128 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab75fedf-4a8e-44fb-916c-1b0c188278ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:36.077408 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75fedf-4a8e-44fb-916c-1b0c188278ac None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:36.077895 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75fedf-4a8e-44fb-916c-1b0c188278ac None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:36.078318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab75fedf-4a8e-44fb-916c-1b0c188278ac None admin] PATH_INFO: `/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/faf778e713b54a638c1db4a5db6180b6/roles/ec841a43f9924fa1b2209bc5ecea5e04` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:36.088215 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:36.096003 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75fedf-4a8e-44fb-916c-1b0c188278ac None admin] RBAC: Authorizing `identity:create_grant(project_id=bc48914270df4db996bd2a4b5b9fc777, user_id=faf778e713b54a638c1db4a5db6180b6, role_id=ec841a43f9924fa1b2209bc5ecea5e04)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:36.097493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab75fedf-4a8e-44fb-916c-1b0c188278ac None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:36.114868 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 412/832] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:48:35 2022] PUT /identity/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/faf778e713b54a638c1db4a5db6180b6/roles/ec841a43f9924fa1b2209bc5ecea5e04 => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 22:48:36.377408 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfc67e-dfb7-4631-947f-354ca80a40b8 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:36.377871 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfc67e-dfb7-4631-947f-354ca80a40b8 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:36.378258 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cfc67e-dfb7-4631-947f-354ca80a40b8 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:36.380589 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 421/833] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:36.385649 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b77d6-8696-4380-ad45-3b3f54ff5c8e None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:36.386169 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b77d6-8696-4380-ad45-3b3f54ff5c8e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:36.386737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115b77d6-8696-4380-ad45-3b3f54ff5c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:36.404704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-115b77d6-8696-4380-ad45-3b3f54ff5c8e None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:36.406216 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-115b77d6-8696-4380-ad45-3b3f54ff5c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:36.479914 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 413/834] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:36 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:48:36.503732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfc67e-dfb7-4631-947f-354ca80a40b8 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:36.522036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:36.615049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cfc67e-dfb7-4631-947f-354ca80a40b8 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:36.650312 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ead39-b330-4aca-b363-99026186b051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:36.651418 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ead39-b330-4aca-b363-99026186b051 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:36.651923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ead39-b330-4aca-b363-99026186b051 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:36.652400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496ead39-b330-4aca-b363-99026186b051 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:36.665272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496ead39-b330-4aca-b363-99026186b051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:36.673075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-496ead39-b330-4aca-b363-99026186b051 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:48:36.673075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:36.673075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:36.673075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:36.673075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:36.673075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:36.673075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:36.673075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:36.673075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:36.673075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:36.673075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:36.673075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:36.673075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:36.673075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:36.673075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:36.673075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:36.673989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:36.673989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:36.673989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:36.673989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:36.673989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:36.673989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:36.673989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:36.673989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:36.673989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:36.673989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:36.673989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:36.673989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:36.673989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:36.673989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:36.673989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:36.673989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:36.674841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:36.674841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:36.674841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:36.674841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:36.674841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:36.674841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:36.674841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:36.674841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:36.674841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:36.674841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:36.674841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:48:36.674841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:36.674841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 422/835] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:48:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:36.683674 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b77d6-8696-4380-ad45-3b3f54ff5c8e None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:36.785196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115b77d6-8696-4380-ad45-3b3f54ff5c8e None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:36.811437 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:36.822665 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d73388-5f08-4191-b59a-dcc9502e7a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:36.823352 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d73388-5f08-4191-b59a-dcc9502e7a4a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:36.823657 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d73388-5f08-4191-b59a-dcc9502e7a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:36.823923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d73388-5f08-4191-b59a-dcc9502e7a4a None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:36.824682 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d73388-5f08-4191-b59a-dcc9502e7a4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:36.824983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d73388-5f08-4191-b59a-dcc9502e7a4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:36.826382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d73388-5f08-4191-b59a-dcc9502e7a4a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:36.836374 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 414/836] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:48:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:36.843645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ead39-b330-4aca-b363-99026186b051 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:36.923386 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:36.955155 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496ead39-b330-4aca-b363-99026186b051 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:36.992409 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff100d-b23b-4b02-a4af-535ae5ed798e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:36.993531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff100d-b23b-4b02-a4af-535ae5ed798e None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:36.994037 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff100d-b23b-4b02-a4af-535ae5ed798e None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:36.994508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ff100d-b23b-4b02-a4af-535ae5ed798e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:37.005607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff100d-b23b-4b02-a4af-535ae5ed798e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:37.007352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ff100d-b23b-4b02-a4af-535ae5ed798e None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:37.025066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-88ff100d-b23b-4b02-a4af-535ae5ed798e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:37.025066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:37.025066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:37.025066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:37.025066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:37.025066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:37.025066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:37.025066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:37.025066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:37.025066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:37.025066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:37.025066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:37.025066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:37.025066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:37.025066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:37.025941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:37.025941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:37.025941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:37.025941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:37.025941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:37.025941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:37.025941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:37.025941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:37.025941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:37.025941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:37.025941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:37.025941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:37.025941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:37.025941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:37.025941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:37.025941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:37.025941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:37.026855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:37.026855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:37.026855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:37.026855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:37.029211 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 423/837] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:37.034950 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d73388-5f08-4191-b59a-dcc9502e7a4a None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:37.138256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d73388-5f08-4191-b59a-dcc9502e7a4a None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:37.173002 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3578921d-8ba0-4ac7-9ec9-d83fc7d1188c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:37.174056 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578921d-8ba0-4ac7-9ec9-d83fc7d1188c None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:37.174544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578921d-8ba0-4ac7-9ec9-d83fc7d1188c None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:37.175068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3578921d-8ba0-4ac7-9ec9-d83fc7d1188c None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:37.176613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578921d-8ba0-4ac7-9ec9-d83fc7d1188c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:37.177200 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578921d-8ba0-4ac7-9ec9-d83fc7d1188c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:37.178852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3578921d-8ba0-4ac7-9ec9-d83fc7d1188c None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:37.193397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 415/838] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:37.199278 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff100d-b23b-4b02-a4af-535ae5ed798e None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:37.315326 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ff100d-b23b-4b02-a4af-535ae5ed798e None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:37.351040 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:37.358116 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15211fd2-3e2c-42bf-a12c-943077d82142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:37.359301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15211fd2-3e2c-42bf-a12c-943077d82142 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:37.360214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15211fd2-3e2c-42bf-a12c-943077d82142 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:37.363140 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15211fd2-3e2c-42bf-a12c-943077d82142 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:37.402344 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d8489-6104-41ee-87ba-914d66ec85f8 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:37.403245 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d8489-6104-41ee-87ba-914d66ec85f8 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:37.403721 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25d8489-6104-41ee-87ba-914d66ec85f8 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:37.405607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 416/839] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:37.412042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b668b-51a5-41bc-88d0-46a95927970b None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:37.412662 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b668b-51a5-41bc-88d0-46a95927970b None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:37.413162 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b668b-51a5-41bc-88d0-46a95927970b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:37.433035 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-170b668b-51a5-41bc-88d0-46a95927970b None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:37.434614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-170b668b-51a5-41bc-88d0-46a95927970b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:37.477530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:37.496723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15211fd2-3e2c-42bf-a12c-943077d82142 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:37.498637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15211fd2-3e2c-42bf-a12c-943077d82142 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:37.520613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 417/840] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:37 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:37.520765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-15211fd2-3e2c-42bf-a12c-943077d82142 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 22:48:37.520765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:37.520765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:37.520765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:37.520765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:37.520765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:37.520765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:37.520765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:37.520765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:37.520765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:37.520765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:37.520765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:37.520765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:37.520765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:37.520765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:37.520765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:37.521635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:37.521635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:37.521635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:37.521635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:37.521635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:37.521635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:37.521635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:37.521635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:37.521635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:37.521635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:37.521635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:37.521635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:37.521635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:37.521635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:37.521635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:37.521635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:37.521635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:37.521635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:37.522561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:37.522561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:37.522561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:37.522561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:37.522561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:37.522561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:37.522561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:37.522561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:37.522561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 22:48:37.522561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:37.522561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 424/841] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:48:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:37.526467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b668b-51a5-41bc-88d0-46a95927970b None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:37.545660 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15211fd2-3e2c-42bf-a12c-943077d82142 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:37.662031 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b668b-51a5-41bc-88d0-46a95927970b None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:37.662768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15211fd2-3e2c-42bf-a12c-943077d82142 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:37.698075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74438b-e27e-47a3-b6b3-4eeaee160fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:37.699125 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74438b-e27e-47a3-b6b3-4eeaee160fda None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:37.699717 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74438b-e27e-47a3-b6b3-4eeaee160fda None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:37.700198 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d74438b-e27e-47a3-b6b3-4eeaee160fda None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:37.701380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74438b-e27e-47a3-b6b3-4eeaee160fda None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:37.701933 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74438b-e27e-47a3-b6b3-4eeaee160fda None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:37.703805 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d74438b-e27e-47a3-b6b3-4eeaee160fda None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:37.704532 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99040b5-3077-461b-8668-a401b0c84242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:37.705339 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99040b5-3077-461b-8668-a401b0c84242 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:37.705625 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99040b5-3077-461b-8668-a401b0c84242 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:37.705901 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99040b5-3077-461b-8668-a401b0c84242 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:37.706758 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99040b5-3077-461b-8668-a401b0c84242 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:37.709510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99040b5-3077-461b-8668-a401b0c84242 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:37.723755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-a99040b5-3077-461b-8668-a401b0c84242 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 22:48:37.723755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:37.723755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:37.723755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:37.723755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:37.723755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:37.723755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:37.723755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:37.723755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 22:48:37.723755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:37.723755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:37.723755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:37.723755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:37.723755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:37.723755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:37.723755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:37.723755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:37.724805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:37.724805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:37.724805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:37.724805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:37.724805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:37.724805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:37.724805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:37.724805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:37.724805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:37.724805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:37.724805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:37.724805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:37.724805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:37.724805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:37.724805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:37.724805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:37.725637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:37.725637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:37.725637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:37.725637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:37.725637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:37.725637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:37.725637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:37.725637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:37.725637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:37.725637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:37.725637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:37.725637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:37.725637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 22:48:37.725637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:37.732923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 425/843] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:48:37 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:37.732923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99040b5-3077-461b-8668-a401b0c84242 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:37.733336 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 418/843] 10.222.0.17 () {56 vars in 1135 bytes} [Thu Jun 16 22:48:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:37.734536 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74438b-e27e-47a3-b6b3-4eeaee160fda None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:37.768617 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:37.801248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:37.848303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d74438b-e27e-47a3-b6b3-4eeaee160fda None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:37.867170 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99040b5-3077-461b-8668-a401b0c84242 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:37.885301 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250171ce-a071-4ac3-afad-53cf538fbb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:37.886365 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250171ce-a071-4ac3-afad-53cf538fbb41 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:37.886921 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250171ce-a071-4ac3-afad-53cf538fbb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:37.887392 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250171ce-a071-4ac3-afad-53cf538fbb41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:37.909601 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd896d-6756-40f5-9da6-bf482c7c7ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:37.910280 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd896d-6756-40f5-9da6-bf482c7c7ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:37.910738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd896d-6756-40f5-9da6-bf482c7c7ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:37.910908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dd896d-6756-40f5-9da6-bf482c7c7ad9 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:37.911616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd896d-6756-40f5-9da6-bf482c7c7ad9 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:37.912433 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd896d-6756-40f5-9da6-bf482c7c7ad9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:37.914042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dd896d-6756-40f5-9da6-bf482c7c7ad9 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:37.922362 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 426/844] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:48:37 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:37.926554 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd896d-6756-40f5-9da6-bf482c7c7ad9 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:37.981790 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250171ce-a071-4ac3-afad-53cf538fbb41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:37.983299 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250171ce-a071-4ac3-afad-53cf538fbb41 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:38.017701 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:38.062999 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dd896d-6756-40f5-9da6-bf482c7c7ad9 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:38.081230 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:38.099549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 419/845] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:48:37 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:48:38.105446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250171ce-a071-4ac3-afad-53cf538fbb41 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:38.108220 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8c9201-388e-4780-a876-b5adaac038d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:38.109736 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c9201-388e-4780-a876-b5adaac038d0 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:38.110036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c9201-388e-4780-a876-b5adaac038d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:38.110322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8c9201-388e-4780-a876-b5adaac038d0 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:38.111077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c9201-388e-4780-a876-b5adaac038d0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:38.111391 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c9201-388e-4780-a876-b5adaac038d0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:38.112763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8c9201-388e-4780-a876-b5adaac038d0 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:38.124747 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 427/846] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:48:37 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:38.241447 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250171ce-a071-4ac3-afad-53cf538fbb41 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:38.262256 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea57852-fd20-4640-87df-1cfd0ae5c799 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:38.262565 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea57852-fd20-4640-87df-1cfd0ae5c799 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:38.262871 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea57852-fd20-4640-87df-1cfd0ae5c799 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:38.264566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 428/847] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:48:38.270513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb12b8-b753-4647-879d-12850df64399 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:38.270853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb12b8-b753-4647-879d-12850df64399 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:38.271115 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cefb12b8-b753-4647-879d-12850df64399 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:38.271640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:38.281546 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adac1c4d-bb80-43ce-afbd-9a6bc2cd6b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:38.282368 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac1c4d-bb80-43ce-afbd-9a6bc2cd6b79 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:38.282680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac1c4d-bb80-43ce-afbd-9a6bc2cd6b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:38.283505 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adac1c4d-bb80-43ce-afbd-9a6bc2cd6b79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:38.289130 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-cefb12b8-b753-4647-879d-12850df64399 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:38.290457 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-cefb12b8-b753-4647-879d-12850df64399 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:38.295378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac1c4d-bb80-43ce-afbd-9a6bc2cd6b79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:38.296842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adac1c4d-bb80-43ce-afbd-9a6bc2cd6b79 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:38.314930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-adac1c4d-bb80-43ce-afbd-9a6bc2cd6b79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:38.314930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:38.314930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:38.314930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:38.314930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:38.314930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:38.314930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:38.314930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:38.314930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:38.314930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:38.314930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:38.314930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:38.314930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:38.314930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:38.314930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:38.315855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:38.315855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:38.315855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:38.315855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:38.315855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:38.315855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:38.315855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:38.315855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:38.315855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:38.315855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:38.315855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:38.315855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:38.315855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:38.315855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:38.315855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:38.315855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:38.315855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:38.317272 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:38.317272 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:38.317272 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:38.317272 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:38.317272 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 420/848] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:38.318438 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29345f0e-758e-4455-b4fe-51c1f653b64c None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:38.318764 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29345f0e-758e-4455-b4fe-51c1f653b64c None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:38.319048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29345f0e-758e-4455-b4fe-51c1f653b64c None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:38.320758 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 421/849] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:38.322303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29345f0e-758e-4455-b4fe-51c1f653b64c None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:38.373189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 429/850] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:38.374566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92109a9-be53-49ee-80a7-21ea216b3bc5 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:38.374918 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92109a9-be53-49ee-80a7-21ea216b3bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:38.375329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92109a9-be53-49ee-80a7-21ea216b3bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:38.398217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a92109a9-be53-49ee-80a7-21ea216b3bc5 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:38.399437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a92109a9-be53-49ee-80a7-21ea216b3bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:38.446311 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29345f0e-758e-4455-b4fe-51c1f653b64c None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:38.472810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 430/851] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:38.474100 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92109a9-be53-49ee-80a7-21ea216b3bc5 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:38.487201 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6007a5a-02d6-4409-b1a5-882654054ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:38.488413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6007a5a-02d6-4409-b1a5-882654054ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:38.489094 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6007a5a-02d6-4409-b1a5-882654054ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:38.489377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6007a5a-02d6-4409-b1a5-882654054ff2 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:38.490418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6007a5a-02d6-4409-b1a5-882654054ff2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:38.490754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6007a5a-02d6-4409-b1a5-882654054ff2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:38.492406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6007a5a-02d6-4409-b1a5-882654054ff2 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:38.508183 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 422/852] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:38.509201 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6007a5a-02d6-4409-b1a5-882654054ff2 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:38.519322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:38.555234 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:38.587489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92109a9-be53-49ee-80a7-21ea216b3bc5 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:38.621101 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6007a5a-02d6-4409-b1a5-882654054ff2 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:38.625328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb9324-f889-4cac-9dd1-e421d49b5086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:38.626194 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb9324-f889-4cac-9dd1-e421d49b5086 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:38.626607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb9324-f889-4cac-9dd1-e421d49b5086 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:38.626899 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb9324-f889-4cac-9dd1-e421d49b5086 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:38.627554 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb9324-f889-4cac-9dd1-e421d49b5086 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:38.628906 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fb9324-f889-4cac-9dd1-e421d49b5086 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:38.642269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-a6fb9324-f889-4cac-9dd1-e421d49b5086 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 22:48:38.642269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:38.642269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:38.642269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:38.642269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:38.642269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:38.642269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:38.642269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:38.642269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 22:48:38.642269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:38.642269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:38.642269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:38.642269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:38.642269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:38.642269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:38.642269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:38.643445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:38.643445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:38.643445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:38.643445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:38.643445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:38.643445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:38.643445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:38.643445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:38.643445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:38.643445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:38.643445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:38.643445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:38.643445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:38.643445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:38.643445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:38.643445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:38.644539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:38.644539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:38.644539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:38.644539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:38.644539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:38.644539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:38.644539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:38.644539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:38.644539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:38.644539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:38.644539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:38.644539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:38.644539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:38.644539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 22:48:38.644539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:38.650286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 431/853] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jun 16 22:48:38 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:38.650286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb9324-f889-4cac-9dd1-e421d49b5086 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:38.667267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbff30-3f91-422b-92c2-0bea9ead3633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:38.668182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbff30-3f91-422b-92c2-0bea9ead3633 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:38.668476 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbff30-3f91-422b-92c2-0bea9ead3633 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:38.668758 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbff30-3f91-422b-92c2-0bea9ead3633 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:38.681683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dbff30-3f91-422b-92c2-0bea9ead3633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:38.690385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-92dbff30-3f91-422b-92c2-0bea9ead3633 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:38.690385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:38.690385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:38.690385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:38.690385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:38.690385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:38.690385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:38.690385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:38.690385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:38.690385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:38.690385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:38.690385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:38.690385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:38.690385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:38.690385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:38.690385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:38.691333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:38.691333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:38.691333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:38.691333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:38.691333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:38.691333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:38.691333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:38.691333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:38.691333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:38.691333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:38.691333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:38.691333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:38.691333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:38.691333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:38.691333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:38.691333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:38.692752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:38.692752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:38.692752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:38.692752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:38.692752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:38.692752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:38.692752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:38.692752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:38.692752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:38.692752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:38.692752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:48:38.692752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:38.692752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 423/854] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:48:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:38.697692 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbff30-3f91-422b-92c2-0bea9ead3633 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:38.767077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb9324-f889-4cac-9dd1-e421d49b5086 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:38.768658 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:38.802676 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:38.804471 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c199f08-d5bf-4643-b041-c3db36d9ca7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:38.805444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c199f08-d5bf-4643-b041-c3db36d9ca7e None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:38.805631 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c199f08-d5bf-4643-b041-c3db36d9ca7e None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:38.805898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c199f08-d5bf-4643-b041-c3db36d9ca7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:38.806022 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbff30-3f91-422b-92c2-0bea9ead3633 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:38.813189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c199f08-d5bf-4643-b041-c3db36d9ca7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:38.814650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c199f08-d5bf-4643-b041-c3db36d9ca7e None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:38.829166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-2c199f08-d5bf-4643-b041-c3db36d9ca7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:38.829166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:38.829166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:38.829166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:38.829166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:38.829166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:38.829166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:38.829166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:38.829166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:38.829166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:38.829166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:38.829166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:38.829166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:38.829166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:38.829166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:38.830042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:38.830042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:38.830042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:38.830042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:38.830042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:38.830042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:38.830042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:38.830042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:38.830042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:38.830042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:38.830042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:38.830042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:38.830042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:38.830042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:38.830042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:38.830042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:38.830042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:38.830961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:38.830961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:38.830961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:38.830961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:38.830961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:38.830961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:38.830961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:38.830961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:38.830961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 432/855] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:38.831442 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c199f08-d5bf-4643-b041-c3db36d9ca7e None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:38.848350 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7ced5-5555-456e-9fc5-db7fffa50fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:38.849029 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7ced5-5555-456e-9fc5-db7fffa50fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:38.849319 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7ced5-5555-456e-9fc5-db7fffa50fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:38.849600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a7ced5-5555-456e-9fc5-db7fffa50fe2 None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:38.850314 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a7ced5-5555-456e-9fc5-db7fffa50fe2 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:38.850653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a7ced5-5555-456e-9fc5-db7fffa50fe2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:38.852037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a7ced5-5555-456e-9fc5-db7fffa50fe2 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:38.856742 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 424/856] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 22:48:38 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:38.857521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7ced5-5555-456e-9fc5-db7fffa50fe2 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:38.940411 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c199f08-d5bf-4643-b041-c3db36d9ca7e None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:38.963932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a7ced5-5555-456e-9fc5-db7fffa50fe2 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:38.978681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a4ad8-96bc-4d7b-a2cd-41d99820c76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:38.979586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a4ad8-96bc-4d7b-a2cd-41d99820c76d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:38.980175 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a4ad8-96bc-4d7b-a2cd-41d99820c76d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:38.980799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8a4ad8-96bc-4d7b-a2cd-41d99820c76d None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:38.981555 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a4ad8-96bc-4d7b-a2cd-41d99820c76d None admin] RBAC: Adding query filter params (name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:38.981866 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a4ad8-96bc-4d7b-a2cd-41d99820c76d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:38.983272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8a4ad8-96bc-4d7b-a2cd-41d99820c76d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:39.002739 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 433/857] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:48:38 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:39.002739 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a4ad8-96bc-4d7b-a2cd-41d99820c76d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:39.003005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5465bba-b490-47c4-bacb-e57d27861898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:39.003005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5465bba-b490-47c4-bacb-e57d27861898 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:39.003005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5465bba-b490-47c4-bacb-e57d27861898 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:39.003005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5465bba-b490-47c4-bacb-e57d27861898 None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:39.003279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5465bba-b490-47c4-bacb-e57d27861898 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:39.003279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5465bba-b490-47c4-bacb-e57d27861898 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:39.004737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5465bba-b490-47c4-bacb-e57d27861898 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:39.029225 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:39.109760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8a4ad8-96bc-4d7b-a2cd-41d99820c76d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:39.137873 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 425/858] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:39.137873 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5465bba-b490-47c4-bacb-e57d27861898 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:39.151335 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c62c1d-b9df-4b7a-b57f-78b5bf903557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:39.152014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c62c1d-b9df-4b7a-b57f-78b5bf903557 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:39.152306 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c62c1d-b9df-4b7a-b57f-78b5bf903557 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:39.152840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c62c1d-b9df-4b7a-b57f-78b5bf903557 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:39.153558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c62c1d-b9df-4b7a-b57f-78b5bf903557 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:39.153875 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c62c1d-b9df-4b7a-b57f-78b5bf903557 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:39.155294 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c62c1d-b9df-4b7a-b57f-78b5bf903557 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:39.159971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 434/859] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 22:48:38 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:39.160966 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:39.161196 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c62c1d-b9df-4b7a-b57f-78b5bf903557 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:39.280904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:39.294024 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5465bba-b490-47c4-bacb-e57d27861898 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:39.299966 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c62c1d-b9df-4b7a-b57f-78b5bf903557 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:39.346348 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c59f0-d78d-4804-99b5-c43c2da3e2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:39.347208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c59f0-d78d-4804-99b5-c43c2da3e2fb None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:39.347492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c59f0-d78d-4804-99b5-c43c2da3e2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:39.347904 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6c59f0-d78d-4804-99b5-c43c2da3e2fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:39.349383 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0981c97-0f5f-4220-9645-d9d95d1debfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:39.350258 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0981c97-0f5f-4220-9645-d9d95d1debfd None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:39.350903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0981c97-0f5f-4220-9645-d9d95d1debfd None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:39.351991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0981c97-0f5f-4220-9645-d9d95d1debfd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:39.352098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0981c97-0f5f-4220-9645-d9d95d1debfd None admin] RBAC: Adding query filter params (role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=bef518c9430e4274b1066241f821f4a1) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:39.352358 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0981c97-0f5f-4220-9645-d9d95d1debfd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:39.355744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0981c97-0f5f-4220-9645-d9d95d1debfd None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:39.359720 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c59f0-d78d-4804-99b5-c43c2da3e2fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:39.360976 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6c59f0-d78d-4804-99b5-c43c2da3e2fb None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:39.362058 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 435/860] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:48:39 2022] GET /identity/v3/role_assignments?user.id=bef518c9430e4274b1066241f821f4a1&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:39.380621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-2e6c59f0-d78d-4804-99b5-c43c2da3e2fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:39.380621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:39.380621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:39.380621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:39.380621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:39.380621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:39.380621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:39.380621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:39.380621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:39.380621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:39.380621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:39.380621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:39.380621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:39.380621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:39.380621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:39.381600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:39.381600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:39.381600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:39.381600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:39.381600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:39.381600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:39.381600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:39.381600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:39.381600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:39.381600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:39.381600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:39.381600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:39.381600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:39.381600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:39.381600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:39.381600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:39.381600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:39.382512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:39.382512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:39.382512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:39.382512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:39.382512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 426/861] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:39.387639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0981c97-0f5f-4220-9645-d9d95d1debfd None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:39.512839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0981c97-0f5f-4220-9645-d9d95d1debfd None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:39.550634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0354b-0e7c-4b2b-a924-fe4c7c780743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:39.551783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0354b-0e7c-4b2b-a924-fe4c7c780743 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:39.552068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0354b-0e7c-4b2b-a924-fe4c7c780743 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:39.552347 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec0354b-0e7c-4b2b-a924-fe4c7c780743 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:39.553076 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0354b-0e7c-4b2b-a924-fe4c7c780743 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:39.553386 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0354b-0e7c-4b2b-a924-fe4c7c780743 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:39.554783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec0354b-0e7c-4b2b-a924-fe4c7c780743 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:39.568247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 436/862] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:39.573693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c59f0-d78d-4804-99b5-c43c2da3e2fb None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:39.657361 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:39.683488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6c59f0-d78d-4804-99b5-c43c2da3e2fb None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:39.717871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543905f1-82d8-4c1d-9194-aa04e71e0bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:39.718725 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543905f1-82d8-4c1d-9194-aa04e71e0bee None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:39.719012 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543905f1-82d8-4c1d-9194-aa04e71e0bee None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:39.719290 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543905f1-82d8-4c1d-9194-aa04e71e0bee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:39.727169 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543905f1-82d8-4c1d-9194-aa04e71e0bee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:39.728615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543905f1-82d8-4c1d-9194-aa04e71e0bee None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:39.743811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-543905f1-82d8-4c1d-9194-aa04e71e0bee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 22:48:39.743811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:39.743811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:39.743811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:39.743811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:39.743811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:39.743811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:39.743811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:39.743811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:39.743811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:39.743811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:39.743811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:39.743811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:39.743811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:39.743811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:39.743811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:39.744780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:39.744780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:39.744780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:39.744780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:39.744780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:39.744780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:39.744780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:39.744780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:39.744780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:39.744780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:39.744780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:39.744780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:39.744780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:39.744780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:39.744780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:39.744780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:39.744780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:39.744780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:39.745727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:39.745727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:39.745727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:39.745727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:39.745727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:39.745727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:39.745727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:39.745727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:39.745727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 22:48:39.745727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:39.745727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 427/863] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 22:48:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:39.752393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0354b-0e7c-4b2b-a924-fe4c7c780743 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:39.768516 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:39.862107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec0354b-0e7c-4b2b-a924-fe4c7c780743 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:39.897932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188d5ec-ec17-45ba-9b32-e3357a3419e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:39.898645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188d5ec-ec17-45ba-9b32-e3357a3419e9 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:39.899006 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188d5ec-ec17-45ba-9b32-e3357a3419e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:39.899293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188d5ec-ec17-45ba-9b32-e3357a3419e9 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:39.900135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188d5ec-ec17-45ba-9b32-e3357a3419e9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:39.900450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188d5ec-ec17-45ba-9b32-e3357a3419e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:39.901876 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1188d5ec-ec17-45ba-9b32-e3357a3419e9 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:39.921407 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 437/864] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jun 16 22:48:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:39.927677 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543905f1-82d8-4c1d-9194-aa04e71e0bee None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:40.030293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543905f1-82d8-4c1d-9194-aa04e71e0bee None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:40.069230 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa178a3-b1b5-4e60-8a86-9a91f2a93f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:40.069879 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa178a3-b1b5-4e60-8a86-9a91f2a93f9b None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:40.070162 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa178a3-b1b5-4e60-8a86-9a91f2a93f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:40.070439 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa178a3-b1b5-4e60-8a86-9a91f2a93f9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:40.073917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:40.140054 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa178a3-b1b5-4e60-8a86-9a91f2a93f9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:40.141406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa178a3-b1b5-4e60-8a86-9a91f2a93f9b None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:40.242922 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 428/865] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:48:39 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:48:40.248705 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188d5ec-ec17-45ba-9b32-e3357a3419e9 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:40.316176 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a8f16-433a-4915-abd8-ccaefde4fa95 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:40.316558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a8f16-433a-4915-abd8-ccaefde4fa95 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:40.316811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5a8f16-433a-4915-abd8-ccaefde4fa95 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:40.319816 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 429/866] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:48:40.324643 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896fbbf-ae61-4985-8978-400f93bba3b0 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:40.324976 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896fbbf-ae61-4985-8978-400f93bba3b0 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:40.325214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6896fbbf-ae61-4985-8978-400f93bba3b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:40.328420 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:40.345883 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-6896fbbf-ae61-4985-8978-400f93bba3b0 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:40.347119 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6896fbbf-ae61-4985-8978-400f93bba3b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:40.360490 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188d5ec-ec17-45ba-9b32-e3357a3419e9 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:40.377370 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:40.399704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f81aff-4505-44fc-8436-762642b138a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:40.400541 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f81aff-4505-44fc-8436-762642b138a2 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:40.400835 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f81aff-4505-44fc-8436-762642b138a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:40.401116 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f81aff-4505-44fc-8436-762642b138a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:40.412590 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f81aff-4505-44fc-8436-762642b138a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:40.414035 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f81aff-4505-44fc-8436-762642b138a2 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:40.430774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 430/867] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:48:40.433327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-d7f81aff-4505-44fc-8436-762642b138a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:40.433327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:40.433327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:40.433327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:40.433327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:40.433327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:40.433327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:40.433327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:40.433327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:40.433327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:40.433327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:40.433327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:40.433327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:40.433327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:40.433327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:40.434325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:40.434325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:40.434325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:40.434325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:40.434325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:40.434325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:40.434325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:40.434325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:40.434325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:40.434325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:40.434325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:40.434325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:40.434325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:40.434325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:40.434325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:40.434325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:40.434325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:40.435246 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:40.435246 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:40.435246 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:40.435246 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:40.435246 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 438/868] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:40.441282 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896fbbf-ae61-4985-8978-400f93bba3b0 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:40.455677 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f81aff-4505-44fc-8436-762642b138a2 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:40.547428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6896fbbf-ae61-4985-8978-400f93bba3b0 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:40.585885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81b06a-c8da-4361-a03e-9f1bb0aa4c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:40.586885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81b06a-c8da-4361-a03e-9f1bb0aa4c67 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:40.587020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81b06a-c8da-4361-a03e-9f1bb0aa4c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:40.587206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef81b06a-c8da-4361-a03e-9f1bb0aa4c67 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:40.588006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81b06a-c8da-4361-a03e-9f1bb0aa4c67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:40.588312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81b06a-c8da-4361-a03e-9f1bb0aa4c67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:40.589685 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef81b06a-c8da-4361-a03e-9f1bb0aa4c67 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:40.598486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f81aff-4505-44fc-8436-762642b138a2 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:40.606913 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 431/869] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:40.608029 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81b06a-c8da-4361-a03e-9f1bb0aa4c67 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:40.628653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:40.638153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:40.648494 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689887b-8b8b-42db-b7bc-3c82125c0287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:40.649473 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689887b-8b8b-42db-b7bc-3c82125c0287 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:40.650017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689887b-8b8b-42db-b7bc-3c82125c0287 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:40.650515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6689887b-8b8b-42db-b7bc-3c82125c0287 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:40.651481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689887b-8b8b-42db-b7bc-3c82125c0287 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:40.653828 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6689887b-8b8b-42db-b7bc-3c82125c0287 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:40.678630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 439/870] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 22:48:40 2022] POST /identity/v3/services => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:48:40.731336 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef81b06a-c8da-4361-a03e-9f1bb0aa4c67 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:40.778023 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f99ab7-d19f-4cf0-adab-55a35aac49f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:40.778876 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f99ab7-d19f-4cf0-adab-55a35aac49f9 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:40.779164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f99ab7-d19f-4cf0-adab-55a35aac49f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:40.779443 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f99ab7-d19f-4cf0-adab-55a35aac49f9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:40.787847 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f99ab7-d19f-4cf0-adab-55a35aac49f9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:40.789390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f99ab7-d19f-4cf0-adab-55a35aac49f9 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:40.805055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-f6f99ab7-d19f-4cf0-adab-55a35aac49f9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:40.805055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:40.805055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:40.805055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:40.805055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:40.805055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:40.805055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:40.805055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:40.805055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:40.805055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:40.805055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:40.805055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:40.805055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:40.805055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:40.805055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:40.806034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:40.806034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:40.806034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:40.806034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:40.806034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:40.806034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:40.806034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:40.806034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:40.806034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:40.806034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:40.806034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:40.806034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:40.806034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:40.806034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:40.806034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:40.806034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:40.806034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:40.807380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:40.807380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:40.807380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:40.807380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:40.807380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:40.807380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:40.807380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:40.807380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:40.810753 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 432/871] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:40.812435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689887b-8b8b-42db-b7bc-3c82125c0287 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:40.919907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6689887b-8b8b-42db-b7bc-3c82125c0287 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:40.954757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff6e708-b787-4bb1-a6ae-5c875e61c0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:40.955533 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6e708-b787-4bb1-a6ae-5c875e61c0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:40.955826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6e708-b787-4bb1-a6ae-5c875e61c0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:40.956107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff6e708-b787-4bb1-a6ae-5c875e61c0e6 None admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:40.956901 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6e708-b787-4bb1-a6ae-5c875e61c0e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:40.957789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6e708-b787-4bb1-a6ae-5c875e61c0e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:40.959262 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff6e708-b787-4bb1-a6ae-5c875e61c0e6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:40.973094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 440/872] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:40.978986 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f99ab7-d19f-4cf0-adab-55a35aac49f9 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:41.067154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:41.088107 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f99ab7-d19f-4cf0-adab-55a35aac49f9 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:41.124621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b87c458-d06a-4a75-80bf-84b2e1e3ca8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:41.125291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87c458-d06a-4a75-80bf-84b2e1e3ca8f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:41.125581 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87c458-d06a-4a75-80bf-84b2e1e3ca8f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:41.125865 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b87c458-d06a-4a75-80bf-84b2e1e3ca8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:41.126728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87c458-d06a-4a75-80bf-84b2e1e3ca8f None admin] RBAC: Adding query filter params (role.id=ec841a43f9924fa1b2209bc5ecea5e04, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=faf778e713b54a638c1db4a5db6180b6) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:41.127051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87c458-d06a-4a75-80bf-84b2e1e3ca8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:41.128488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b87c458-d06a-4a75-80bf-84b2e1e3ca8f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:41.135083 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 433/873] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:48:40 2022] GET /identity/v3/role_assignments?user.id=faf778e713b54a638c1db4a5db6180b6&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=ec841a43f9924fa1b2209bc5ecea5e04 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:41.400735 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb11388-f195-4ec8-b520-0cba926a285a None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:41.401230 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb11388-f195-4ec8-b520-0cba926a285a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:41.401412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb11388-f195-4ec8-b520-0cba926a285a None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:41.403172 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 441/874] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:41.408259 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8412c58-4f28-4fca-bc97-53cac3b2e30e None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:41.408562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8412c58-4f28-4fca-bc97-53cac3b2e30e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:41.408847 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8412c58-4f28-4fca-bc97-53cac3b2e30e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:41.426679 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-d8412c58-4f28-4fca-bc97-53cac3b2e30e None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:41.427874 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d8412c58-4f28-4fca-bc97-53cac3b2e30e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:41.500235 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 434/875] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:41 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:41.523519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb11388-f195-4ec8-b520-0cba926a285a None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:41.573848 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:41.643424 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb11388-f195-4ec8-b520-0cba926a285a None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:41.650441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ff494-22a9-4fcc-b76a-1271d51e4e89 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:41.650763 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ff494-22a9-4fcc-b76a-1271d51e4e89 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:41.651051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01ff494-22a9-4fcc-b76a-1271d51e4e89 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:41.652979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 435/876] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:41.658027 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb452e-b477-4395-977f-6d00429a33f8 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:41.658327 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb452e-b477-4395-977f-6d00429a33f8 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:41.658631 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfb452e-b477-4395-977f-6d00429a33f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:41.677118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-2cfb452e-b477-4395-977f-6d00429a33f8 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:41.678318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-2cfb452e-b477-4395-977f-6d00429a33f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:41.682848 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197625aa-231f-45f0-8c94-254e1f066566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:41.683716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197625aa-231f-45f0-8c94-254e1f066566 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:41.684147 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197625aa-231f-45f0-8c94-254e1f066566 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:41.684570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197625aa-231f-45f0-8c94-254e1f066566 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:41.685461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197625aa-231f-45f0-8c94-254e1f066566 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:41.687088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197625aa-231f-45f0-8c94-254e1f066566 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:41.706459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 442/877] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 22:48:41 2022] POST /identity/v3/services => generated 269 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:48:41.768529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 436/878] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:41 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:48:41.791965 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197625aa-231f-45f0-8c94-254e1f066566 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:41.901759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197625aa-231f-45f0-8c94-254e1f066566 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:41.903748 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:41.944379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb157cfa-924d-49d6-a4d3-8ca2c33b6d3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:41.945196 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb157cfa-924d-49d6-a4d3-8ca2c33b6d3f None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:41.945482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb157cfa-924d-49d6-a4d3-8ca2c33b6d3f None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:41.945759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb157cfa-924d-49d6-a4d3-8ca2c33b6d3f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:41.946420 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb157cfa-924d-49d6-a4d3-8ca2c33b6d3f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:41.947834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb157cfa-924d-49d6-a4d3-8ca2c33b6d3f None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:41.960237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-eb157cfa-924d-49d6-a4d3-8ca2c33b6d3f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 22:48:41.960237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:41.960237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:41.960237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:41.960237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:41.960237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:41.960237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:41.960237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:41.960237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 22:48:41.960237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:41.960237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:41.960237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:41.960237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:41.960237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:41.960237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:41.960237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:41.960237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:41.961284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:41.961284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:41.961284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:41.961284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:41.961284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:41.961284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:41.961284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:41.961284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:41.961284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:41.961284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:41.961284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:41.961284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:41.961284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:41.961284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:41.961284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:41.961284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:41.962145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:41.962145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:41.962145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:41.962145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:41.962145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:41.962145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:41.962145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:41.962145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:41.962145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:41.962145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:41.962145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:41.962145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:41.962145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 22:48:41.962145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:41.962145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 443/879] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 22:48:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:41.967727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb452e-b477-4395-977f-6d00429a33f8 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:41.970094 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:42.079641 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfb452e-b477-4395-977f-6d00429a33f8 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:42.115629 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f469f6-7a18-4f68-be25-aa20c0bc5f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:42.116620 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f469f6-7a18-4f68-be25-aa20c0bc5f3c None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:42.117132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f469f6-7a18-4f68-be25-aa20c0bc5f3c None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:42.117613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f469f6-7a18-4f68-be25-aa20c0bc5f3c None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:42.118538 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f469f6-7a18-4f68-be25-aa20c0bc5f3c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:42.119077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f469f6-7a18-4f68-be25-aa20c0bc5f3c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:42.120644 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f469f6-7a18-4f68-be25-aa20c0bc5f3c None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:42.125927 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 437/880] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 22:48:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:42.132218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb157cfa-924d-49d6-a4d3-8ca2c33b6d3f None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:42.247448 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb157cfa-924d-49d6-a4d3-8ca2c33b6d3f None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:42.283280 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b49df-a213-4186-be1b-6dbce05f924b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:42.283952 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b49df-a213-4186-be1b-6dbce05f924b None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:42.284238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b49df-a213-4186-be1b-6dbce05f924b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:42.284514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b49df-a213-4186-be1b-6dbce05f924b None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:42.285761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b49df-a213-4186-be1b-6dbce05f924b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:42.286072 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b49df-a213-4186-be1b-6dbce05f924b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:42.287497 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b49df-a213-4186-be1b-6dbce05f924b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:42.292551 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 444/881] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 22:48:42 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:42.999430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fac782-98e2-4117-ad9b-bcc01b0e2bd7 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:42.999944 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fac782-98e2-4117-ad9b-bcc01b0e2bd7 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:43.000337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fac782-98e2-4117-ad9b-bcc01b0e2bd7 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:43.002058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 438/882] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:43.007935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49158f-7a02-42f5-b897-3a6050a80d46 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:43.008410 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49158f-7a02-42f5-b897-3a6050a80d46 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:43.008935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e49158f-7a02-42f5-b897-3a6050a80d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:43.026902 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-0e49158f-7a02-42f5-b897-3a6050a80d46 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:43.028348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-0e49158f-7a02-42f5-b897-3a6050a80d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:43.053941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:43.102604 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 445/883] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:43.127984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fac782-98e2-4117-ad9b-bcc01b0e2bd7 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:43.198189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:43.238618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fac782-98e2-4117-ad9b-bcc01b0e2bd7 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:43.269984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600eb5ea-47ef-4e3c-bddd-ddd5eb8d3147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:43.270764 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600eb5ea-47ef-4e3c-bddd-ddd5eb8d3147 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:43.271056 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600eb5ea-47ef-4e3c-bddd-ddd5eb8d3147 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:43.271338 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600eb5ea-47ef-4e3c-bddd-ddd5eb8d3147 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:43.271987 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600eb5ea-47ef-4e3c-bddd-ddd5eb8d3147 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:43.273424 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600eb5ea-47ef-4e3c-bddd-ddd5eb8d3147 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:43.285984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-600eb5ea-47ef-4e3c-bddd-ddd5eb8d3147 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 22:48:43.285984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:43.285984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:43.285984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:43.285984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:43.285984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:43.285984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:43.285984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:43.285984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 22:48:43.285984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:43.285984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:43.285984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:43.285984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:43.285984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:43.285984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:43.285984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:43.285984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:43.287068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:43.287068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:43.287068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:43.287068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:43.287068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:43.287068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:43.287068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:43.287068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:43.287068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:43.287068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:43.287068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:43.287068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:43.287068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:43.287068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:43.287068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:43.287068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:43.288105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:43.288105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:43.288105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:43.288105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:43.288105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:43.288105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:43.288105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:43.288105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:43.288105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:43.288105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:43.288105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:43.288105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:43.288105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 22:48:43.288105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:43.288105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 439/884] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:48:43 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:43.293434 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49158f-7a02-42f5-b897-3a6050a80d46 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:43.357032 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca9f7d-6e23-4be8-920c-bb647e1b12f2 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:43.357354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca9f7d-6e23-4be8-920c-bb647e1b12f2 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:43.357589 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ca9f7d-6e23-4be8-920c-bb647e1b12f2 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:43.359213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 440/885] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:43.366302 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61f8be-70ad-4141-a976-e04ad014924a None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:43.366566 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61f8be-70ad-4141-a976-e04ad014924a None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:43.366871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61f8be-70ad-4141-a976-e04ad014924a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:43.384708 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-6e61f8be-70ad-4141-a976-e04ad014924a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:43.385855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6e61f8be-70ad-4141-a976-e04ad014924a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:43.404331 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e49158f-7a02-42f5-b897-3a6050a80d46 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:43.441603 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a257525-0e1d-484f-a799-20ffa477a880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:43.442286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a257525-0e1d-484f-a799-20ffa477a880 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:43.442610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a257525-0e1d-484f-a799-20ffa477a880 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:43.443066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a257525-0e1d-484f-a799-20ffa477a880 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:43.443614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a257525-0e1d-484f-a799-20ffa477a880 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:43.443928 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a257525-0e1d-484f-a799-20ffa477a880 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:43.445300 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a257525-0e1d-484f-a799-20ffa477a880 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:43.450577 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 446/886] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:48:43 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:43.456040 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a257525-0e1d-484f-a799-20ffa477a880 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:43.474734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 441/887] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:43 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:43.486585 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:43.497658 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61f8be-70ad-4141-a976-e04ad014924a None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:43.560763 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:43.582898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a257525-0e1d-484f-a799-20ffa477a880 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:43.607556 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61f8be-70ad-4141-a976-e04ad014924a None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:43.621444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2a202-c1c9-4178-bc7a-24ad3d0f0c09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:43.627629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2a202-c1c9-4178-bc7a-24ad3d0f0c09 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:43.627629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2a202-c1c9-4178-bc7a-24ad3d0f0c09 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:43.627629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae2a202-c1c9-4178-bc7a-24ad3d0f0c09 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:43.627991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2a202-c1c9-4178-bc7a-24ad3d0f0c09 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:43.628211 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2a202-c1c9-4178-bc7a-24ad3d0f0c09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:43.630114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae2a202-c1c9-4178-bc7a-24ad3d0f0c09 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:43.640634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 447/888] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:48:43 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:43.641526 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2a202-c1c9-4178-bc7a-24ad3d0f0c09 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:43.656039 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259dba1f-d3ad-4fb6-b590-13234909f9b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:43.657273 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dba1f-d3ad-4fb6-b590-13234909f9b1 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:43.657930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dba1f-d3ad-4fb6-b590-13234909f9b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:43.658407 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259dba1f-d3ad-4fb6-b590-13234909f9b1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:43.673092 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259dba1f-d3ad-4fb6-b590-13234909f9b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:43.683502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-259dba1f-d3ad-4fb6-b590-13234909f9b1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 22:48:43.683502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:43.683502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:43.683502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:43.683502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:43.683502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:43.683502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:43.683502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:43.683502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:43.683502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:43.683502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:43.683502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:43.683502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:43.683502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:43.683502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:43.683502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:43.684456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:43.684456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:43.684456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:43.684456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:43.684456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:43.684456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:43.684456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:43.684456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:43.684456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:43.684456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:43.684456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:43.684456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:43.684456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:43.684456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:43.684456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:43.684456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:43.685342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:43.685342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:43.685342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:43.685342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:43.685342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:43.685342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:43.685342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:43.685342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:43.685342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:43.685342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:43.685342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 22:48:43.685342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:43.690685 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 442/889] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jun 16 22:48:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:43.694153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259dba1f-d3ad-4fb6-b590-13234909f9b1 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:43.743445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:43.758252 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae2a202-c1c9-4178-bc7a-24ad3d0f0c09 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:43.795728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70448f40-da3c-4cd0-8201-8d8c0ead9aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:43.796741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70448f40-da3c-4cd0-8201-8d8c0ead9aef None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:43.797091 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70448f40-da3c-4cd0-8201-8d8c0ead9aef None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:43.797376 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70448f40-da3c-4cd0-8201-8d8c0ead9aef None admin] PATH_INFO: `/v3/endpoints` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:43.798169 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70448f40-da3c-4cd0-8201-8d8c0ead9aef None admin] RBAC: Adding query filter params (interface=public, service_id=c2c36aa005b04624ba459ae0b9716fb3, region_id=RegionOne) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:43.798475 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70448f40-da3c-4cd0-8201-8d8c0ead9aef None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:43.801121 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70448f40-da3c-4cd0-8201-8d8c0ead9aef None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:43.810656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 448/890] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jun 16 22:48:43 2022] GET /identity/v3/endpoints?service_id=c2c36aa005b04624ba459ae0b9716fb3&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:43.812989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70448f40-da3c-4cd0-8201-8d8c0ead9aef None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:43.827712 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259dba1f-d3ad-4fb6-b590-13234909f9b1 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:43.852407 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:43.870276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd6a2a-c7ca-448a-8360-8e8f793761b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:43.871127 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd6a2a-c7ca-448a-8360-8e8f793761b0 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:43.871417 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd6a2a-c7ca-448a-8360-8e8f793761b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:43.871698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bd6a2a-c7ca-448a-8360-8e8f793761b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:43.872449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd6a2a-c7ca-448a-8360-8e8f793761b0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:43.872758 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd6a2a-c7ca-448a-8360-8e8f793761b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:43.874687 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bd6a2a-c7ca-448a-8360-8e8f793761b0 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:43.889949 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 443/891] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jun 16 22:48:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:43.891104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd6a2a-c7ca-448a-8360-8e8f793761b0 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:43.939087 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70448f40-da3c-4cd0-8201-8d8c0ead9aef None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:43.980264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938b7308-8f40-43fd-8af4-da0a9ae7c5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:43.980949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b7308-8f40-43fd-8af4-da0a9ae7c5ce None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:43.981233 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b7308-8f40-43fd-8af4-da0a9ae7c5ce None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:43.981510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938b7308-8f40-43fd-8af4-da0a9ae7c5ce None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:43.982195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b7308-8f40-43fd-8af4-da0a9ae7c5ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:43.983579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938b7308-8f40-43fd-8af4-da0a9ae7c5ce None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:43.990721 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 449/892] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jun 16 22:48:43 2022] GET /identity/v3/services => generated 1268 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 22:48:44.014972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2a69d-5002-4517-9afc-3b2739b10da1 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:44.015605 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2a69d-5002-4517-9afc-3b2739b10da1 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:44.015605 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2a69d-5002-4517-9afc-3b2739b10da1 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:44.021299 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 450/893] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:48:44.022042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bd6a2a-c7ca-448a-8360-8e8f793761b0 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:44.023261 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81090242-db66-497a-92c0-142ac78e71eb None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:44.023557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81090242-db66-497a-92c0-142ac78e71eb None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:44.023844 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81090242-db66-497a-92c0-142ac78e71eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:44.042348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-81090242-db66-497a-92c0-142ac78e71eb None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:44.043609 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-81090242-db66-497a-92c0-142ac78e71eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:44.059889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:44.064102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f90bd2-5df7-40e8-80dd-8ac81a66a4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:44.065440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f90bd2-5df7-40e8-80dd-8ac81a66a4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:44.065876 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f90bd2-5df7-40e8-80dd-8ac81a66a4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:44.067063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f90bd2-5df7-40e8-80dd-8ac81a66a4a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:44.087172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f90bd2-5df7-40e8-80dd-8ac81a66a4a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:44.089001 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f90bd2-5df7-40e8-80dd-8ac81a66a4a5 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:44.115196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-85f90bd2-5df7-40e8-80dd-8ac81a66a4a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:44.115196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:44.115196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:44.115196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:44.115196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:44.115196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:44.115196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:44.115196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:44.115196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:44.115196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:44.115196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:44.115196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:44.115196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:44.115196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:44.115196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:44.116364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:44.116364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:44.116364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:44.116364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:44.116364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:44.116364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:44.116364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:44.116364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:44.116364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:44.116364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:44.116364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:44.116364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:44.116364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:44.116364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:44.116364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:44.116364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:44.116364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:44.117308 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:44.117308 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:44.117308 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:44.117308 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:44.117308 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 444/894] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:44.122180 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f90bd2-5df7-40e8-80dd-8ac81a66a4a5 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:44.129846 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 451/895] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:44.160008 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81090242-db66-497a-92c0-142ac78e71eb None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:44.162992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:44.252212 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f90bd2-5df7-40e8-80dd-8ac81a66a4a5 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:44.287253 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f698cd-f514-4fc4-af2e-be1c32184111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:44.287925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f698cd-f514-4fc4-af2e-be1c32184111 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:44.288208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f698cd-f514-4fc4-af2e-be1c32184111 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:44.288488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f698cd-f514-4fc4-af2e-be1c32184111 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:44.289208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f698cd-f514-4fc4-af2e-be1c32184111 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:44.289518 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f698cd-f514-4fc4-af2e-be1c32184111 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:44.290906 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f698cd-f514-4fc4-af2e-be1c32184111 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:44.295408 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81090242-db66-497a-92c0-142ac78e71eb None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:44.304135 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 445/896] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:44.309531 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f698cd-f514-4fc4-af2e-be1c32184111 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:44.334182 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b33fa0-d46c-49c9-a11b-ffb300cd6d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:44.335096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b33fa0-d46c-49c9-a11b-ffb300cd6d86 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:44.335392 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b33fa0-d46c-49c9-a11b-ffb300cd6d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:44.335675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b33fa0-d46c-49c9-a11b-ffb300cd6d86 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:44.336333 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b33fa0-d46c-49c9-a11b-ffb300cd6d86 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:44.337897 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b33fa0-d46c-49c9-a11b-ffb300cd6d86 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:44.351759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-f1b33fa0-d46c-49c9-a11b-ffb300cd6d86 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 22:48:44.351759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:44.351759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:44.351759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:44.351759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:44.351759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:44.351759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:44.351759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:44.351759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 22:48:44.351759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:44.351759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:44.351759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:44.351759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:44.351759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:44.351759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:44.351759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:44.352767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:44.352767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:44.352767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:44.352767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:44.352767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:44.352767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:44.352767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:44.352767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:44.352767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:44.352767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:44.352767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:44.352767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:44.352767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:44.352767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:44.352767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:44.352767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:44.353607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:44.353607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:44.353607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:44.353607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:44.353607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:44.353607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:44.353607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:44.353607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:44.353607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:44.353607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:44.353607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:44.353607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:44.353607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:44.353607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 22:48:44.353607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:44.358181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 452/897] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jun 16 22:48:44 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:44.359507 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b33fa0-d46c-49c9-a11b-ffb300cd6d86 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:44.374027 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:44.407457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:44.415788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f698cd-f514-4fc4-af2e-be1c32184111 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:44.454210 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09441376-7699-47f3-9a5a-274a9406b83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:44.455132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09441376-7699-47f3-9a5a-274a9406b83b None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:44.455755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09441376-7699-47f3-9a5a-274a9406b83b None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:44.456031 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09441376-7699-47f3-9a5a-274a9406b83b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:44.464217 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09441376-7699-47f3-9a5a-274a9406b83b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:44.465661 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09441376-7699-47f3-9a5a-274a9406b83b None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:44.482636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b33fa0-d46c-49c9-a11b-ffb300cd6d86 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:44.487533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-09441376-7699-47f3-9a5a-274a9406b83b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 22:48:44.487533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:44.487533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:44.487533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:44.487533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:44.487533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:44.487533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:44.487533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:44.487533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:44.487533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:44.487533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:44.487533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:44.487533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:44.487533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:44.487533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:44.487533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:44.489044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:44.489044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:44.489044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:44.489044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:44.489044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:44.489044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:44.489044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:44.489044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:44.489044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:44.489044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:44.489044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:44.489044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:44.489044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:44.489044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:44.489044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:44.489044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:44.489044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:44.489044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:44.490734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:44.490734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:44.490734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:44.490734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:44.490734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:44.490734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:44.490734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:44.490734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:44.490734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 22:48:44.490734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:44.494122 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 446/898] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 22:48:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:44.495709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348eea1c-d259-47d8-8a99-abec295c154b None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:44.495889 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348eea1c-d259-47d8-8a99-abec295c154b None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:44.496146 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348eea1c-d259-47d8-8a99-abec295c154b None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:44.498556 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 447/899] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:44.501226 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348eea1c-d259-47d8-8a99-abec295c154b None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:44.542620 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae39c9aa-1308-4aa4-ab1c-fa16a7c3833d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:44.542620 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39c9aa-1308-4aa4-ab1c-fa16a7c3833d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:44.542620 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39c9aa-1308-4aa4-ab1c-fa16a7c3833d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:44.542620 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae39c9aa-1308-4aa4-ab1c-fa16a7c3833d None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:44.543065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39c9aa-1308-4aa4-ab1c-fa16a7c3833d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:44.543065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39c9aa-1308-4aa4-ab1c-fa16a7c3833d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:44.551051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae39c9aa-1308-4aa4-ab1c-fa16a7c3833d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:44.562630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 453/900] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 22:48:44 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:44.562630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191f338-f5a4-45e2-ae3c-5cf3f00ac7c9 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:44.562630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191f338-f5a4-45e2-ae3c-5cf3f00ac7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:44.562630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4191f338-f5a4-45e2-ae3c-5cf3f00ac7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:44.578615 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-4191f338-f5a4-45e2-ae3c-5cf3f00ac7c9 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:44.578615 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-4191f338-f5a4-45e2-ae3c-5cf3f00ac7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:44.622621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348eea1c-d259-47d8-8a99-abec295c154b None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:44.649801 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 454/901] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:44 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:44.650749 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191f338-f5a4-45e2-ae3c-5cf3f00ac7c9 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:44.655759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:44.682947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b6bc4-640f-4417-b8de-ff16e3e59b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:44.683630 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b6bc4-640f-4417-b8de-ff16e3e59b7f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:44.683970 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b6bc4-640f-4417-b8de-ff16e3e59b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:44.684241 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c89b6bc4-640f-4417-b8de-ff16e3e59b7f None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:44.685144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b6bc4-640f-4417-b8de-ff16e3e59b7f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:44.685464 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b6bc4-640f-4417-b8de-ff16e3e59b7f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:44.686940 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c89b6bc4-640f-4417-b8de-ff16e3e59b7f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:44.712939 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 448/902] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jun 16 22:48:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:44.713837 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b6bc4-640f-4417-b8de-ff16e3e59b7f None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:44.729353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:44.772071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4191f338-f5a4-45e2-ae3c-5cf3f00ac7c9 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:44.818986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c33b48-95ef-4124-b741-5a398b7d2d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:44.820077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c33b48-95ef-4124-b741-5a398b7d2d8a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:44.820765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c33b48-95ef-4124-b741-5a398b7d2d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:44.821356 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c33b48-95ef-4124-b741-5a398b7d2d8a None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:44.822287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c33b48-95ef-4124-b741-5a398b7d2d8a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:44.822733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c33b48-95ef-4124-b741-5a398b7d2d8a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:44.824328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c33b48-95ef-4124-b741-5a398b7d2d8a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:44.830326 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 455/903] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 22:48:44 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:44.836525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c33b48-95ef-4124-b741-5a398b7d2d8a None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:44.841554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c89b6bc4-640f-4417-b8de-ff16e3e59b7f None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:44.876894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df87cac-20d7-4d3c-a2b8-7e588b120128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:44.878631 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df87cac-20d7-4d3c-a2b8-7e588b120128 None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:44.879063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df87cac-20d7-4d3c-a2b8-7e588b120128 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:44.879497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df87cac-20d7-4d3c-a2b8-7e588b120128 None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:44.880493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df87cac-20d7-4d3c-a2b8-7e588b120128 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:44.881880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df87cac-20d7-4d3c-a2b8-7e588b120128 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:44.907344 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 449/904] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 22:48:44 2022] POST /identity/v3/services => generated 241 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:48:44.911857 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df87cac-20d7-4d3c-a2b8-7e588b120128 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:44.920993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:44.958282 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c33b48-95ef-4124-b741-5a398b7d2d8a None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:45.003041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401068d4-ed5e-4983-91e0-5010bb34d306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:45.004248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401068d4-ed5e-4983-91e0-5010bb34d306 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:45.004538 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401068d4-ed5e-4983-91e0-5010bb34d306 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:45.004818 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401068d4-ed5e-4983-91e0-5010bb34d306 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:45.025122 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:45.062968 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df87cac-20d7-4d3c-a2b8-7e588b120128 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:45.079574 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401068d4-ed5e-4983-91e0-5010bb34d306 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:45.081051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401068d4-ed5e-4983-91e0-5010bb34d306 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:45.107838 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e456a-a8ac-4937-8f41-a08746a1be22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:45.108514 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e456a-a8ac-4937-8f41-a08746a1be22 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:45.108800 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e456a-a8ac-4937-8f41-a08746a1be22 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:45.109082 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48e456a-a8ac-4937-8f41-a08746a1be22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:45.109878 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e456a-a8ac-4937-8f41-a08746a1be22 None admin] RBAC: Adding query filter params (interface=public, service_id=28b55e8d23a44d68b06611adca010fd4, region_id=RegionOne) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:45.110196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e456a-a8ac-4937-8f41-a08746a1be22 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:45.111613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48e456a-a8ac-4937-8f41-a08746a1be22 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:45.118563 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 450/905] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jun 16 22:48:44 2022] GET /identity/v3/endpoints?service_id=28b55e8d23a44d68b06611adca010fd4&interface=public®ion_id=RegionOne => generated 194 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:45.125443 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e456a-a8ac-4937-8f41-a08746a1be22 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:45.159946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:45.205974 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 456/906] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:48:44 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:48:45.211422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401068d4-ed5e-4983-91e0-5010bb34d306 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:45.242258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48e456a-a8ac-4937-8f41-a08746a1be22 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:45.276627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02787a-8e31-41c6-836c-7c45b7a500bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:45.277834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02787a-8e31-41c6-836c-7c45b7a500bb None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:45.278262 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02787a-8e31-41c6-836c-7c45b7a500bb None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:45.278761 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d02787a-8e31-41c6-836c-7c45b7a500bb None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:45.279631 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02787a-8e31-41c6-836c-7c45b7a500bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:45.281173 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d02787a-8e31-41c6-836c-7c45b7a500bb None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:45.283233 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:45.294621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 451/907] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jun 16 22:48:45 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 22:48:45.334870 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401068d4-ed5e-4983-91e0-5010bb34d306 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:45.372690 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a39f8f-a556-45b6-a68e-bd477eacba2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:45.373505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a39f8f-a556-45b6-a68e-bd477eacba2e None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:45.373790 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a39f8f-a556-45b6-a68e-bd477eacba2e None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:45.374068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a39f8f-a556-45b6-a68e-bd477eacba2e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:45.386049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a39f8f-a556-45b6-a68e-bd477eacba2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:45.387656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a39f8f-a556-45b6-a68e-bd477eacba2e None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:45.408852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-81a39f8f-a556-45b6-a68e-bd477eacba2e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:45.408852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:45.408852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:45.408852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:45.408852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:45.408852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:45.408852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:45.408852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:45.408852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:45.408852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:45.408852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:45.408852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:45.408852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:45.408852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:45.408852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:45.409840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:45.409840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:45.409840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:45.409840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:45.409840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:45.409840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:45.409840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:45.409840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:45.409840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:45.409840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:45.409840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:45.409840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:45.409840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:45.409840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:45.409840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:45.409840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:45.409840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:45.410816 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:45.410816 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:45.410816 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:45.410816 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:45.410816 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 457/908] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:45.417119 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02787a-8e31-41c6-836c-7c45b7a500bb None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:45.524864 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d02787a-8e31-41c6-836c-7c45b7a500bb None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:45.560126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c117bd4-6c47-4696-9e02-9f0732718fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:45.560799 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c117bd4-6c47-4696-9e02-9f0732718fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:45.561080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c117bd4-6c47-4696-9e02-9f0732718fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:45.561357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c117bd4-6c47-4696-9e02-9f0732718fa8 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:45.562075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c117bd4-6c47-4696-9e02-9f0732718fa8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:45.562383 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c117bd4-6c47-4696-9e02-9f0732718fa8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:45.563785 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c117bd4-6c47-4696-9e02-9f0732718fa8 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:45.576548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 452/909] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:45.582605 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a39f8f-a556-45b6-a68e-bd477eacba2e None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:45.601840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:45.687162 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a39f8f-a556-45b6-a68e-bd477eacba2e None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:45.724204 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485e446-6891-4fa3-94a2-59b3830d07a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:45.725015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485e446-6891-4fa3-94a2-59b3830d07a4 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:45.725298 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485e446-6891-4fa3-94a2-59b3830d07a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:45.725573 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9485e446-6891-4fa3-94a2-59b3830d07a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:45.732855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485e446-6891-4fa3-94a2-59b3830d07a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:45.734285 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9485e446-6891-4fa3-94a2-59b3830d07a4 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:45.748455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-9485e446-6891-4fa3-94a2-59b3830d07a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:45.748455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:45.748455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:45.748455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:45.748455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:45.748455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:45.748455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:45.748455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:45.748455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:45.748455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:45.748455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:45.748455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:45.748455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:45.748455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:45.748455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:45.749406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:45.749406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:45.749406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:45.749406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:45.749406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:45.749406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:45.749406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:45.749406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:45.749406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:45.749406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:45.749406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:45.749406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:45.749406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:45.749406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:45.749406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:45.749406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:45.749406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:45.750850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:45.750850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:45.750850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:45.750850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:45.750850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:45.750850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:45.750850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:45.750850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:45.750850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 458/910] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:45.757247 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c117bd4-6c47-4696-9e02-9f0732718fa8 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:45.819256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:45.865787 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c117bd4-6c47-4696-9e02-9f0732718fa8 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:45.902336 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97df0334-47af-43b2-9a3b-b839aef7136f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:45.903058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df0334-47af-43b2-9a3b-b839aef7136f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:45.903474 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df0334-47af-43b2-9a3b-b839aef7136f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:45.903751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97df0334-47af-43b2-9a3b-b839aef7136f None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:45.904533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df0334-47af-43b2-9a3b-b839aef7136f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:45.904844 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df0334-47af-43b2-9a3b-b839aef7136f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:45.906259 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97df0334-47af-43b2-9a3b-b839aef7136f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:45.918582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 453/911] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:45.924610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485e446-6891-4fa3-94a2-59b3830d07a4 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:46.003221 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:46.028519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9485e446-6891-4fa3-94a2-59b3830d07a4 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:46.062724 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9510ff62-2647-4bd1-9cdb-ab7da492b9a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:46.063438 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510ff62-2647-4bd1-9cdb-ab7da492b9a2 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:46.063720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510ff62-2647-4bd1-9cdb-ab7da492b9a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:46.064052 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9510ff62-2647-4bd1-9cdb-ab7da492b9a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:46.064882 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510ff62-2647-4bd1-9cdb-ab7da492b9a2 None admin] RBAC: Adding query filter params (role.id=c6bedc2399d04c78a942f96283ed4560, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=faf778e713b54a638c1db4a5db6180b6) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:46.065193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510ff62-2647-4bd1-9cdb-ab7da492b9a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:46.067161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9510ff62-2647-4bd1-9cdb-ab7da492b9a2 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:46.073082 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 459/912] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:48:45 2022] GET /identity/v3/role_assignments?user.id=faf778e713b54a638c1db4a5db6180b6&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=c6bedc2399d04c78a942f96283ed4560 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:46.289877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eaa495-0d25-452f-a8e1-7398ef5bc84c None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:46.290362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eaa495-0d25-452f-a8e1-7398ef5bc84c None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:46.290362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45eaa495-0d25-452f-a8e1-7398ef5bc84c None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:46.291990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 454/913] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:46.297262 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb6426-a549-4bec-acf3-ed8ee2367741 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:46.297663 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb6426-a549-4bec-acf3-ed8ee2367741 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:46.297834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bb6426-a549-4bec-acf3-ed8ee2367741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:46.315526 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-02bb6426-a549-4bec-acf3-ed8ee2367741 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:46.316767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-02bb6426-a549-4bec-acf3-ed8ee2367741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:46.385272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 460/914] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:46 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:46.408757 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eaa495-0d25-452f-a8e1-7398ef5bc84c None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:46.515787 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45eaa495-0d25-452f-a8e1-7398ef5bc84c None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:46.539581 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:46.562914 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d0121-d160-478f-928c-e764d36debff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:46.563650 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d0121-d160-478f-928c-e764d36debff None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:46.563933 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d0121-d160-478f-928c-e764d36debff None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:46.564213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6d0121-d160-478f-928c-e764d36debff None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:46.564868 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d0121-d160-478f-928c-e764d36debff None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:46.566851 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6d0121-d160-478f-928c-e764d36debff None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:46.579613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-ac6d0121-d160-478f-928c-e764d36debff None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 22:48:46.579613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:46.579613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:46.579613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:46.579613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:46.579613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:46.579613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:46.579613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:46.579613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 22:48:46.579613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:46.579613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:46.579613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:46.579613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:46.579613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:46.579613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:46.579613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:46.579613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:46.580594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:46.580594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:46.580594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:46.580594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:46.580594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:46.580594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:46.580594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:46.580594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:46.580594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:46.580594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:46.580594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:46.580594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:46.580594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:46.580594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:46.580594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:46.580594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:46.581990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:46.581990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:46.581990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:46.581990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:46.581990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:46.581990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:46.581990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:46.581990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:46.581990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:46.581990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:46.581990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:46.581990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:46.581990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 22:48:46.581990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:46.581990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 455/915] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:48:46 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:46.586834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb6426-a549-4bec-acf3-ed8ee2367741 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:46.663163 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:46.694875 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02bb6426-a549-4bec-acf3-ed8ee2367741 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:46.730394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae6a98-16b5-49fc-a9e8-811b5ef1b981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:46.731101 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae6a98-16b5-49fc-a9e8-811b5ef1b981 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:46.731444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae6a98-16b5-49fc-a9e8-811b5ef1b981 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:46.731725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae6a98-16b5-49fc-a9e8-811b5ef1b981 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:46.732437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae6a98-16b5-49fc-a9e8-811b5ef1b981 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:46.732749 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae6a98-16b5-49fc-a9e8-811b5ef1b981 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:46.734166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae6a98-16b5-49fc-a9e8-811b5ef1b981 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:46.739485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 461/916] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:48:46 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:46.745030 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d0121-d160-478f-928c-e764d36debff None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:46.854157 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6d0121-d160-478f-928c-e764d36debff None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:46.889814 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0141803c-a7e5-448b-9ca7-1a373e83cb58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:46.890866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141803c-a7e5-448b-9ca7-1a373e83cb58 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:46.891351 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141803c-a7e5-448b-9ca7-1a373e83cb58 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:46.891837 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0141803c-a7e5-448b-9ca7-1a373e83cb58 None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:46.892747 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141803c-a7e5-448b-9ca7-1a373e83cb58 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:46.893248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141803c-a7e5-448b-9ca7-1a373e83cb58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:46.895380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0141803c-a7e5-448b-9ca7-1a373e83cb58 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:46.900824 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 456/917] 10.222.0.17 () {56 vars in 1114 bytes} [Thu Jun 16 22:48:46 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:46.910630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae6a98-16b5-49fc-a9e8-811b5ef1b981 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:47.008271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae6a98-16b5-49fc-a9e8-811b5ef1b981 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:47.045353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4596cd5-acdd-4696-8bdb-b1122a68a0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:47.046569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4596cd5-acdd-4696-8bdb-b1122a68a0e7 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:47.047142 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4596cd5-acdd-4696-8bdb-b1122a68a0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:47.047650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4596cd5-acdd-4696-8bdb-b1122a68a0e7 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:47.048560 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4596cd5-acdd-4696-8bdb-b1122a68a0e7 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:47.049934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4596cd5-acdd-4696-8bdb-b1122a68a0e7 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:47.054898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:47.075042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 462/918] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jun 16 22:48:46 2022] POST /identity/v3/endpoints => generated 346 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:48:47.227137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186443c-c7c2-478a-ab22-70f71b1304f2 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:47.227793 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186443c-c7c2-478a-ab22-70f71b1304f2 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:47.234210 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1186443c-c7c2-478a-ab22-70f71b1304f2 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:47.236504 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 457/919] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:47 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:47.242438 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27361923-fbb9-4abb-a7fb-8f0e67c57055 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:47.242981 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27361923-fbb9-4abb-a7fb-8f0e67c57055 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:47.243443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27361923-fbb9-4abb-a7fb-8f0e67c57055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:47.266366 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-27361923-fbb9-4abb-a7fb-8f0e67c57055 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:47.267847 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-27361923-fbb9-4abb-a7fb-8f0e67c57055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:47.344224 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 463/920] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:48:47.367295 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186443c-c7c2-478a-ab22-70f71b1304f2 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:47.420693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:47.474233 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1186443c-c7c2-478a-ab22-70f71b1304f2 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:47.509004 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c9068c-5929-4ec7-ace2-ecad89c9f964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:47.509740 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9068c-5929-4ec7-ace2-ecad89c9f964 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:47.510306 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9068c-5929-4ec7-ace2-ecad89c9f964 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:47.510583 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c9068c-5929-4ec7-ace2-ecad89c9f964 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:47.511264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9068c-5929-4ec7-ace2-ecad89c9f964 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:47.512670 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c9068c-5929-4ec7-ace2-ecad89c9f964 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:47.524975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-57c9068c-5929-4ec7-ace2-ecad89c9f964 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 22:48:47.524975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:47.524975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:47.524975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:47.524975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:47.524975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:47.524975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:47.524975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:47.524975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 22:48:47.524975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:47.524975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:47.524975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:47.524975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:47.524975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:47.524975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:47.524975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:47.524975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:47.525971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:47.525971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:47.525971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:47.525971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:47.525971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:47.525971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:47.525971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:47.525971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:47.525971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:47.525971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:47.525971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:47.525971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:47.525971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:47.525971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:47.525971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:47.525971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:47.526830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:47.526830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:47.526830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:47.526830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:47.526830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:47.526830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:47.526830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:47.526830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:47.526830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:47.526830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:47.526830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:47.526830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:47.526830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 22:48:47.526830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:47.526830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 458/921] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 22:48:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:47.534276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27361923-fbb9-4abb-a7fb-8f0e67c57055 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:47.638340 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0224c-15a0-410f-aede-35e555b74ea2 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:47.638690 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0224c-15a0-410f-aede-35e555b74ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:47.638933 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0224c-15a0-410f-aede-35e555b74ea2 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:47.640507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 459/922] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:47.649268 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27361923-fbb9-4abb-a7fb-8f0e67c57055 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:47.652360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f5b57-ec5e-43b1-b3d2-92235b77a6d5 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:47.652652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f5b57-ec5e-43b1-b3d2-92235b77a6d5 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:47.652931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053f5b57-ec5e-43b1-b3d2-92235b77a6d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:47.668941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:47.681053 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-053f5b57-ec5e-43b1-b3d2-92235b77a6d5 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:47.682542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-053f5b57-ec5e-43b1-b3d2-92235b77a6d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:47.688431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40968058-984f-48b0-8c0c-cb81f1fa9157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:47.689106 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40968058-984f-48b0-8c0c-cb81f1fa9157 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:47.689389 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40968058-984f-48b0-8c0c-cb81f1fa9157 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:47.689666 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40968058-984f-48b0-8c0c-cb81f1fa9157 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:47.690391 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40968058-984f-48b0-8c0c-cb81f1fa9157 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:47.690714 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40968058-984f-48b0-8c0c-cb81f1fa9157 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:47.692632 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40968058-984f-48b0-8c0c-cb81f1fa9157 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:47.702067 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 464/923] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 22:48:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:47.703300 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40968058-984f-48b0-8c0c-cb81f1fa9157 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:47.766762 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 460/924] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:47.797624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f5b57-ec5e-43b1-b3d2-92235b77a6d5 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:47.844911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:47.853004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40968058-984f-48b0-8c0c-cb81f1fa9157 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:47.905249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a8821-c0c3-4895-a296-0edc7bb3449d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:47.906882 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a8821-c0c3-4895-a296-0edc7bb3449d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:47.907370 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a8821-c0c3-4895-a296-0edc7bb3449d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:47.907827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87a8821-c0c3-4895-a296-0edc7bb3449d None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:47.908780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053f5b57-ec5e-43b1-b3d2-92235b77a6d5 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:47.909069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a8821-c0c3-4895-a296-0edc7bb3449d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:47.910425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a8821-c0c3-4895-a296-0edc7bb3449d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:47.913522 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87a8821-c0c3-4895-a296-0edc7bb3449d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:47.924673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 465/925] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 22:48:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:47.925896 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a8821-c0c3-4895-a296-0edc7bb3449d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:47.961733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d58ecb-3f6f-4f76-9cda-af5a944b3b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:47.964559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d58ecb-3f6f-4f76-9cda-af5a944b3b40 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:47.965083 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d58ecb-3f6f-4f76-9cda-af5a944b3b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:47.965574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d58ecb-3f6f-4f76-9cda-af5a944b3b40 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:47.966451 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d58ecb-3f6f-4f76-9cda-af5a944b3b40 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:47.968070 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d58ecb-3f6f-4f76-9cda-af5a944b3b40 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:47.976103 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:47.989471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-88d58ecb-3f6f-4f76-9cda-af5a944b3b40 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 22:48:47.989471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:47.989471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:47.989471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:47.989471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:47.989471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:47.989471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:47.989471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:47.989471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 22:48:47.989471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:47.989471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:47.989471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:47.989471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:47.989471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:47.989471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:47.989471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:47.990459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:47.990459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:47.990459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:47.990459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:47.990459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:47.990459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:47.990459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:47.990459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:47.990459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:47.990459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:47.990459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:47.990459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:47.990459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:47.990459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:47.990459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:47.990459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:47.991318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:47.991318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:47.991318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:47.991318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:47.991318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:47.991318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:47.991318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:47.991318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:47.991318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:47.991318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:47.991318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:47.991318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:47.991318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:47.991318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 22:48:47.991318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:47.997628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 461/926] 10.222.0.17 () {56 vars in 1132 bytes} [Thu Jun 16 22:48:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:47.997628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d58ecb-3f6f-4f76-9cda-af5a944b3b40 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:48.047244 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87a8821-c0c3-4895-a296-0edc7bb3449d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:48.083037 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08695d20-52eb-42a1-a497-fa57b2a1c06d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:48.083712 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08695d20-52eb-42a1-a497-fa57b2a1c06d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:48.084000 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08695d20-52eb-42a1-a497-fa57b2a1c06d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:48.084282 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08695d20-52eb-42a1-a497-fa57b2a1c06d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:48.085079 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08695d20-52eb-42a1-a497-fa57b2a1c06d None admin] RBAC: Adding query filter params (interface=public, service_id=85a71b72ad414bfd90c542849583a722, region_id=RegionOne) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:48.085455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08695d20-52eb-42a1-a497-fa57b2a1c06d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:48.087387 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08695d20-52eb-42a1-a497-fa57b2a1c06d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:48.099579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 466/927] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jun 16 22:48:47 2022] GET /identity/v3/endpoints?service_id=85a71b72ad414bfd90c542849583a722&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:48.101469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08695d20-52eb-42a1-a497-fa57b2a1c06d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:48.121223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d58ecb-3f6f-4f76-9cda-af5a944b3b40 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:48.129275 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:48.162763 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d2c5a-5748-44c9-ae3c-a68d1f9d830d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:48.164211 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d2c5a-5748-44c9-ae3c-a68d1f9d830d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:48.164818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d2c5a-5748-44c9-ae3c-a68d1f9d830d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:48.165399 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d2c5a-5748-44c9-ae3c-a68d1f9d830d None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:48.166336 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d2c5a-5748-44c9-ae3c-a68d1f9d830d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:48.166914 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d2c5a-5748-44c9-ae3c-a68d1f9d830d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:48.168632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d2c5a-5748-44c9-ae3c-a68d1f9d830d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:48.180661 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 462/928] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 22:48:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:48.182002 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d2c5a-5748-44c9-ae3c-a68d1f9d830d None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:48.214294 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08695d20-52eb-42a1-a497-fa57b2a1c06d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:48.225199 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:48.253499 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0189cfb-8e27-4451-904a-01bd79cd7015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:48.254296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0189cfb-8e27-4451-904a-01bd79cd7015 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:48.254726 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0189cfb-8e27-4451-904a-01bd79cd7015 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:48.255010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0189cfb-8e27-4451-904a-01bd79cd7015 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:48.255697 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0189cfb-8e27-4451-904a-01bd79cd7015 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:48.257171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0189cfb-8e27-4451-904a-01bd79cd7015 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:48.270618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 467/929] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jun 16 22:48:48 2022] GET /identity/v3/services => generated 1497 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 22:48:48.270618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf396174-6396-40d7-b2dd-909ffd87bfec None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:48.270618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf396174-6396-40d7-b2dd-909ffd87bfec None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:48.270618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf396174-6396-40d7-b2dd-909ffd87bfec None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:48.270618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 468/930] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:48.272715 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c12e7-a185-4df0-9268-b9081a46af12 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:48.273102 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c12e7-a185-4df0-9268-b9081a46af12 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:48.273478 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1c12e7-a185-4df0-9268-b9081a46af12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:48.293687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-5d1c12e7-a185-4df0-9268-b9081a46af12 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:48.294992 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5d1c12e7-a185-4df0-9268-b9081a46af12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:48.317185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d2c5a-5748-44c9-ae3c-a68d1f9d830d None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:48.364147 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee1c32-4ab1-486b-b6c3-b501c3df8c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:48.365156 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee1c32-4ab1-486b-b6c3-b501c3df8c6c None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:48.365450 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee1c32-4ab1-486b-b6c3-b501c3df8c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:48.365734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aee1c32-4ab1-486b-b6c3-b501c3df8c6c None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:48.366787 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee1c32-4ab1-486b-b6c3-b501c3df8c6c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:48.367107 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee1c32-4ab1-486b-b6c3-b501c3df8c6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:48.369357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aee1c32-4ab1-486b-b6c3-b501c3df8c6c None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:48.384038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 463/931] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 22:48:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:48.393443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 469/932] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:48 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:48.395926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee1c32-4ab1-486b-b6c3-b501c3df8c6c None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:48.417988 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c12e7-a185-4df0-9268-b9081a46af12 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:48.460715 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:48.548181 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aee1c32-4ab1-486b-b6c3-b501c3df8c6c None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:48.554831 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1c12e7-a185-4df0-9268-b9081a46af12 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:48.558985 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:48.586230 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd725f76-075f-4824-8523-7a770478c8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:48.586838 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd725f76-075f-4824-8523-7a770478c8fb None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:48.587122 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd725f76-075f-4824-8523-7a770478c8fb None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:48.587400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd725f76-075f-4824-8523-7a770478c8fb None admin] PATH_INFO: `/v3/endpoints` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:48.588139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd725f76-075f-4824-8523-7a770478c8fb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:48.589501 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd725f76-075f-4824-8523-7a770478c8fb None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:48.601438 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a391737-6e1a-4ba2-a5eb-f9ea849b1dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:48.602829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a391737-6e1a-4ba2-a5eb-f9ea849b1dd8 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:48.603266 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a391737-6e1a-4ba2-a5eb-f9ea849b1dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:48.603648 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a391737-6e1a-4ba2-a5eb-f9ea849b1dd8 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:48.613254 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 464/933] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jun 16 22:48:48 2022] POST /identity/v3/endpoints => generated 347 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:48:48.620963 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a391737-6e1a-4ba2-a5eb-f9ea849b1dd8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:48.625413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-2a391737-6e1a-4ba2-a5eb-f9ea849b1dd8 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 22:48:48.625413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:48.625413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:48.625413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:48.625413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:48.625413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:48.625413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:48.625413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:48.625413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:48.625413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:48.625413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:48.625413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:48.625413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:48.625413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:48.625413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:48.625413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:48.626439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:48.626439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:48.626439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:48.626439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:48.626439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:48.626439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:48.626439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:48.626439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:48.626439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:48.626439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:48.626439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:48.626439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:48.626439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:48.626439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:48.626439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:48.626439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:48.627746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:48.627746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:48.627746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:48.627746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:48.627746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:48.627746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:48.627746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:48.627746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:48.627746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:48.627746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:48.627746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 22:48:48.627746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:48.627746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 470/934] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jun 16 22:48:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:48.634346 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd725f76-075f-4824-8523-7a770478c8fb None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:48.743670 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:48.764968 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd725f76-075f-4824-8523-7a770478c8fb None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:48.812149 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e8d36-ce28-4dff-9bdf-3d73d6114de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:48.812837 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e8d36-ce28-4dff-9bdf-3d73d6114de8 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:48.813126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e8d36-ce28-4dff-9bdf-3d73d6114de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:48.813407 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6e8d36-ce28-4dff-9bdf-3d73d6114de8 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:48.814161 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e8d36-ce28-4dff-9bdf-3d73d6114de8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:48.814473 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e8d36-ce28-4dff-9bdf-3d73d6114de8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:48.816488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6e8d36-ce28-4dff-9bdf-3d73d6114de8 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:48.825563 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 465/935] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jun 16 22:48:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:48.830872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a391737-6e1a-4ba2-a5eb-f9ea849b1dd8 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:48.935588 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a391737-6e1a-4ba2-a5eb-f9ea849b1dd8 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:48.971767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df0a18-7399-4c0c-84de-2ec76943c28c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:48.972592 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df0a18-7399-4c0c-84de-2ec76943c28c None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:48.972878 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df0a18-7399-4c0c-84de-2ec76943c28c None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:48.973157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83df0a18-7399-4c0c-84de-2ec76943c28c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:48.984289 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df0a18-7399-4c0c-84de-2ec76943c28c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:48.985856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83df0a18-7399-4c0c-84de-2ec76943c28c None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:49.004096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-83df0a18-7399-4c0c-84de-2ec76943c28c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:49.004096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:49.004096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:49.004096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:49.004096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:49.004096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:49.004096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:49.004096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:49.004096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:49.004096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:49.004096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:49.004096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:49.004096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:49.004096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:49.004096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:49.006341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:49.006341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:49.006341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:49.006341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:49.006341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:49.006341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:49.006341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:49.006341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:49.006341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:49.006341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:49.006341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:49.006341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:49.006341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:49.006341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:49.006341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:49.006341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:49.006341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:49.008467 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:49.008467 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:49.008467 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:49.008467 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:49.008467 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 471/936] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:49.011057 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e8d36-ce28-4dff-9bdf-3d73d6114de8 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:49.118941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6e8d36-ce28-4dff-9bdf-3d73d6114de8 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:49.154149 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51ca41-ccdc-4fab-9f61-a9da50296675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:49.154852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51ca41-ccdc-4fab-9f61-a9da50296675 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:49.155138 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51ca41-ccdc-4fab-9f61-a9da50296675 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:49.155420 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa51ca41-ccdc-4fab-9f61-a9da50296675 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:49.156262 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51ca41-ccdc-4fab-9f61-a9da50296675 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:49.156577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51ca41-ccdc-4fab-9f61-a9da50296675 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:49.157939 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa51ca41-ccdc-4fab-9f61-a9da50296675 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:49.170798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 466/937] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:49.179701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df0a18-7399-4c0c-84de-2ec76943c28c None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:49.196247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:49.284228 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83df0a18-7399-4c0c-84de-2ec76943c28c None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:49.319052 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5179e2-424a-4a1d-92f9-1a1625211803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:49.319989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5179e2-424a-4a1d-92f9-1a1625211803 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:49.320276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5179e2-424a-4a1d-92f9-1a1625211803 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:49.320595 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5179e2-424a-4a1d-92f9-1a1625211803 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:49.328323 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5179e2-424a-4a1d-92f9-1a1625211803 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:49.329809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5179e2-424a-4a1d-92f9-1a1625211803 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:49.345109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-ff5179e2-424a-4a1d-92f9-1a1625211803 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 22:48:49.345109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:49.345109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:49.345109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:49.345109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:49.345109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:49.345109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:49.345109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:49.345109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:49.345109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:49.345109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:49.345109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:49.345109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:49.345109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:49.345109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:49.345109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:49.346043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:49.346043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:49.346043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:49.346043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:49.346043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:49.346043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:49.346043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:49.346043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:49.346043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:49.346043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:49.346043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:49.346043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:49.346043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:49.346043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:49.346043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:49.346043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:49.346043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:49.346043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:49.348235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:49.348235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:49.348235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:49.348235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:49.348235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:49.348235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:49.348235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:49.348235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:49.348235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 22:48:49.348235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:49.348235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 472/938] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 22:48:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:49.352560 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51ca41-ccdc-4fab-9f61-a9da50296675 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:49.364836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:49.430078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3ffd9-4185-478f-a8a9-3aac7eb07d4e None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:49.430662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3ffd9-4185-478f-a8a9-3aac7eb07d4e None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:49.430662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d3ffd9-4185-478f-a8a9-3aac7eb07d4e None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:49.433363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 473/939] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:48:49.437520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a803f9-5a86-4e25-a282-5fbe3577673a None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:49.437810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a803f9-5a86-4e25-a282-5fbe3577673a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:49.438089 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a803f9-5a86-4e25-a282-5fbe3577673a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:49.456096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-64a803f9-5a86-4e25-a282-5fbe3577673a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:49.456096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-64a803f9-5a86-4e25-a282-5fbe3577673a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:49.473185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa51ca41-ccdc-4fab-9f61-a9da50296675 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:49.509168 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd815b8-f179-4a83-ad2d-82791b5959e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:49.509852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd815b8-f179-4a83-ad2d-82791b5959e0 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:49.510139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd815b8-f179-4a83-ad2d-82791b5959e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:49.510419 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd815b8-f179-4a83-ad2d-82791b5959e0 None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:49.511400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd815b8-f179-4a83-ad2d-82791b5959e0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:49.511782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd815b8-f179-4a83-ad2d-82791b5959e0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:49.513210 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd815b8-f179-4a83-ad2d-82791b5959e0 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:49.530727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 474/940] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:48:49.532449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 467/941] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jun 16 22:48:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:49.538263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a803f9-5a86-4e25-a282-5fbe3577673a None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:49.554872 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd815b8-f179-4a83-ad2d-82791b5959e0 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:49.555080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:49.632992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:49.647960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a803f9-5a86-4e25-a282-5fbe3577673a None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:49.681951 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd815b8-f179-4a83-ad2d-82791b5959e0 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:49.688018 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372ea20-0aae-401a-aa66-64a2e88562de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:49.688886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372ea20-0aae-401a-aa66-64a2e88562de None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:49.689240 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372ea20-0aae-401a-aa66-64a2e88562de None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:49.689585 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c372ea20-0aae-401a-aa66-64a2e88562de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:49.700170 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372ea20-0aae-401a-aa66-64a2e88562de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:49.701680 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c372ea20-0aae-401a-aa66-64a2e88562de None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:49.719350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-c372ea20-0aae-401a-aa66-64a2e88562de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:49.719350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:49.719350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:49.719350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:49.719350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:49.719350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:49.719350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:49.719350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:49.719350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:49.719350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:49.719350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:49.719350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:49.719350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:49.719350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:49.719350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:49.720245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:49.720245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:49.720245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:49.720245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:49.720245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:49.720245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:49.720245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:49.720245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:49.720245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:49.720245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:49.720245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:49.720245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:49.720245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:49.720245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:49.720245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:49.720245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:49.720245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:49.721121 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:49.721121 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:49.721121 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:49.721121 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:49.725618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 475/942] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:49.726873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372ea20-0aae-401a-aa66-64a2e88562de None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:49.731388 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71042456-e363-4ad5-9dc8-127f4a4f59e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:49.732637 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71042456-e363-4ad5-9dc8-127f4a4f59e7 None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:49.733259 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71042456-e363-4ad5-9dc8-127f4a4f59e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:49.733665 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71042456-e363-4ad5-9dc8-127f4a4f59e7 None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:49.734458 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71042456-e363-4ad5-9dc8-127f4a4f59e7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:49.735999 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71042456-e363-4ad5-9dc8-127f4a4f59e7 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:49.737537 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: WARNING keystone.server.flask.common [None req-71042456-e363-4ad5-9dc8-127f4a4f59e7 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 22:48:49.767652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.sql.core [None req-71042456-e363-4ad5-9dc8-127f4a4f59e7 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 22:48:49.767652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 22:48:49.767652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [parameters: {'id': 'f6a543f4d2df44a48ae3cfd834fa1aac', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 22:48:49.767652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80088) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 22:48:49.788827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-71042456-e363-4ad5-9dc8-127f4a4f59e7 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f6a543f4d2df44a48ae3cfd834fa1aac.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f6a543f4d2df44a48ae3cfd834fa1aac. Jun 16 22:48:49.788827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:49.788827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 22:48:49.788827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 22:48:49.788827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 22:48:49.788827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 22:48:49.788827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 22:48:49.788827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 22:48:49.788827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 22:48:49.788827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application conn.query(q) Jun 16 22:48:49.788827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 22:48:49.788827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 22:48:49.788827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 22:48:49.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application result.read() Jun 16 22:48:49.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 22:48:49.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 22:48:49.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 22:48:49.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 22:48:49.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 22:48:49.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 22:48:49.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 22:48:49.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 22:48:49.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 22:48:49.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:49.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 22:48:49.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:49.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:49.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 22:48:49.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 22:48:49.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 16 22:48:49.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 16 22:48:49.791152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 22:48:49.791152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application next(self.gen) Jun 16 22:48:49.791152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 22:48:49.791152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application yield resource Jun 16 22:48:49.791152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 22:48:49.791152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application next(self.gen) Jun 16 22:48:49.791152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 22:48:49.791152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 22:48:49.791152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 22:48:49.791152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 22:48:49.791152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 22:48:49.791152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise self.value Jun 16 22:48:49.791152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 22:48:49.791152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 22:48:49.791152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 22:48:49.791152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application session.commit() Jun 16 22:48:49.791152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 22:48:49.792325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 22:48:49.792325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 22:48:49.792325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 22:48:49.792325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 22:48:49.792325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self.session.flush() Jun 16 22:48:49.792325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 22:48:49.792325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 22:48:49.792325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 22:48:49.792325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 22:48:49.792325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 22:48:49.792325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application compat.raise_( Jun 16 22:48:49.792325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 22:48:49.792325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception Jun 16 22:48:49.792325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 22:48:49.792325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 22:48:49.792325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 22:48:49.792325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 22:48:49.793206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 22:48:49.793206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 22:48:49.793206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 22:48:49.793206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 22:48:49.793206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 22:48:49.793206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 22:48:49.793206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 22:48:49.793206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 22:48:49.793206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 22:48:49.793206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 22:48:49.793206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 22:48:49.793206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 22:48:49.793206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 22:48:49.793206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 22:48:49.793206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 22:48:49.794009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 22:48:49.794009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 22:48:49.794009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception Jun 16 22:48:49.794009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 22:48:49.794009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 22:48:49.794009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 22:48:49.794009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 22:48:49.794009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 22:48:49.794009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 22:48:49.794009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 22:48:49.794009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application conn.query(q) Jun 16 22:48:49.794009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 22:48:49.794009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 22:48:49.794009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 22:48:49.794009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application result.read() Jun 16 22:48:49.794009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 22:48:49.794009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 22:48:49.795988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 22:48:49.795988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 22:48:49.795988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 22:48:49.795988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 22:48:49.795988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 22:48:49.795988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 22:48:49.795988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 22:48:49.795988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 22:48:49.795988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [parameters: {'id': 'f6a543f4d2df44a48ae3cfd834fa1aac', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 22:48:49.795988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 22:48:49.795988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:49.795988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:49.795988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:49.795988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:49.795988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 16 22:48:49.797823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 16 22:48:49.797823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 22:48:49.797823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 22:48:49.797823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 16 22:48:49.797823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:49.797823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:49.797823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:49.797823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:49.797823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:49.797823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:49.797823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:49.797823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:49.797823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:49.797823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:49.797823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:49.797823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:49.797823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:49.800888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:49.800888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 16 22:48:49.800888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 16 22:48:49.800888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:49.800888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:49.800888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 16 22:48:49.800888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 16 22:48:49.800888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id f6a543f4d2df44a48ae3cfd834fa1aac. Jun 16 22:48:49.800888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:49.800888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 468/943] 10.222.0.17 () {60 vars in 1140 bytes} [Thu Jun 16 22:48:49 2022] POST /identity/v3/projects => generated 262 bytes in 236 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:49.800888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71042456-e363-4ad5-9dc8-127f4a4f59e7 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:49.803495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:49.854870 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c372ea20-0aae-401a-aa66-64a2e88562de None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:49.900255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7045de-3bf5-4992-92c7-bd3610861aaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:49.901380 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7045de-3bf5-4992-92c7-bd3610861aaa None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:49.902164 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7045de-3bf5-4992-92c7-bd3610861aaa None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:49.902684 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7045de-3bf5-4992-92c7-bd3610861aaa None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:49.903672 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7045de-3bf5-4992-92c7-bd3610861aaa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:49.904180 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7045de-3bf5-4992-92c7-bd3610861aaa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:49.905805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7045de-3bf5-4992-92c7-bd3610861aaa None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:49.923106 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 476/944] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:49.924721 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7045de-3bf5-4992-92c7-bd3610861aaa None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:49.925494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71042456-e363-4ad5-9dc8-127f4a4f59e7 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:49.952438 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:49.969255 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2dd563-24e7-479c-b5a4-2ae9ef51d015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:49.970529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dd563-24e7-479c-b5a4-2ae9ef51d015 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:49.970932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dd563-24e7-479c-b5a4-2ae9ef51d015 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:49.971342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2dd563-24e7-479c-b5a4-2ae9ef51d015 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:49.979942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2dd563-24e7-479c-b5a4-2ae9ef51d015 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:49.982307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2dd563-24e7-479c-b5a4-2ae9ef51d015 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:50.003632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-1b2dd563-24e7-479c-b5a4-2ae9ef51d015 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:50.003632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:50.003632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:50.003632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:50.003632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:50.003632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:50.003632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:50.003632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:50.003632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:50.003632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:50.003632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:50.003632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:50.003632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:50.003632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:50.003632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:50.004714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:50.004714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:50.004714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:50.004714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:50.004714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:50.004714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:50.004714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:50.004714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:50.004714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:50.004714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:50.004714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:50.004714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:50.004714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:50.004714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:50.004714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:50.004714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:50.004714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:50.005776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:50.005776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:50.005776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:50.005776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:50.005776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:50.005776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:50.005776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:50.005776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:50.010006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 469/945] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:50.011381 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2dd563-24e7-479c-b5a4-2ae9ef51d015 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:50.059646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7045de-3bf5-4992-92c7-bd3610861aaa None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:50.080882 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:50.107946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8557b1b-69c9-40a6-bac4-b7faa90d6939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:50.109649 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8557b1b-69c9-40a6-bac4-b7faa90d6939 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:50.109941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8557b1b-69c9-40a6-bac4-b7faa90d6939 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:50.110224 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8557b1b-69c9-40a6-bac4-b7faa90d6939 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:50.119705 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8557b1b-69c9-40a6-bac4-b7faa90d6939 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:50.121535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8557b1b-69c9-40a6-bac4-b7faa90d6939 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:50.140883 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2dd563-24e7-479c-b5a4-2ae9ef51d015 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:50.141450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-c8557b1b-69c9-40a6-bac4-b7faa90d6939 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:50.141450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:50.141450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:50.141450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:50.141450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:50.141450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:50.141450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:50.141450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:50.141450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:50.141450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:50.141450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:50.141450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:50.141450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:50.141450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:50.141450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:50.142505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:50.142505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:50.142505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:50.142505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:50.142505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:50.142505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:50.142505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:50.142505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:50.142505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:50.142505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:50.142505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:50.142505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:50.142505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:50.142505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:50.142505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:50.142505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:50.142505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:50.143476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:50.143476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:50.143476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:50.143476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:50.143476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:50.143476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:50.143476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:50.143476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:50.143476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 477/946] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:50.148987 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8557b1b-69c9-40a6-bac4-b7faa90d6939 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:50.184213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5084f31-bec8-478c-bbe6-18178b014451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:50.185215 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5084f31-bec8-478c-bbe6-18178b014451 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:50.185698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5084f31-bec8-478c-bbe6-18178b014451 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:50.186171 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5084f31-bec8-478c-bbe6-18178b014451 None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:50.187160 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5084f31-bec8-478c-bbe6-18178b014451 None admin] RBAC: Adding query filter params (name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:50.187681 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5084f31-bec8-478c-bbe6-18178b014451 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:50.189280 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5084f31-bec8-478c-bbe6-18178b014451 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:50.210288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 470/947] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 22:48:50 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:50.277819 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8557b1b-69c9-40a6-bac4-b7faa90d6939 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:50.317977 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd9017e-b39b-48c4-98c1-190e4621d1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:50.318693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9017e-b39b-48c4-98c1-190e4621d1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:50.318980 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9017e-b39b-48c4-98c1-190e4621d1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:50.319253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd9017e-b39b-48c4-98c1-190e4621d1a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:50.320053 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9017e-b39b-48c4-98c1-190e4621d1a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:50.320363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9017e-b39b-48c4-98c1-190e4621d1a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:50.321919 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd9017e-b39b-48c4-98c1-190e4621d1a2 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:50.341304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 478/948] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:50.343806 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5084f31-bec8-478c-bbe6-18178b014451 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:50.394658 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:50.461584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5084f31-bec8-478c-bbe6-18178b014451 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:50.497857 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabecaf-0271-42f1-bc72-8d666e0e9da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:50.498738 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabecaf-0271-42f1-bc72-8d666e0e9da1 None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:50.499026 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabecaf-0271-42f1-bc72-8d666e0e9da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:50.499344 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabecaf-0271-42f1-bc72-8d666e0e9da1 None admin] PATH_INFO: `/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/faf778e713b54a638c1db4a5db6180b6/roles/c6bedc2399d04c78a942f96283ed4560` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:50.501763 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabecaf-0271-42f1-bc72-8d666e0e9da1 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc48914270df4db996bd2a4b5b9fc777, user_id=faf778e713b54a638c1db4a5db6180b6, role_id=c6bedc2399d04c78a942f96283ed4560)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:50.503763 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabecaf-0271-42f1-bc72-8d666e0e9da1 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:50.505199 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ae705-e9b8-48b4-b070-cfc428eadfff None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:50.505607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ae705-e9b8-48b4-b070-cfc428eadfff None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:50.505783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728ae705-e9b8-48b4-b070-cfc428eadfff None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:50.507157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 479/949] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:48:50.512627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca24e4b-8580-4e77-b808-a5a835ed2e79 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:50.512955 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca24e4b-8580-4e77-b808-a5a835ed2e79 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:50.513241 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca24e4b-8580-4e77-b808-a5a835ed2e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:50.519421 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 471/950] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:48:50 2022] PUT /identity/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/faf778e713b54a638c1db4a5db6180b6/roles/c6bedc2399d04c78a942f96283ed4560 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:48:50.532815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-2ca24e4b-8580-4e77-b808-a5a835ed2e79 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:50.534093 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-2ca24e4b-8580-4e77-b808-a5a835ed2e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:50.569690 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:50.614625 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 480/951] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:48:50.639023 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabecaf-0271-42f1-bc72-8d666e0e9da1 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:50.750279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabecaf-0271-42f1-bc72-8d666e0e9da1 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:50.751841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:50.787336 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71292071-a36e-444f-bb2b-aed9ca3c92de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:50.788155 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71292071-a36e-444f-bb2b-aed9ca3c92de None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:50.788441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71292071-a36e-444f-bb2b-aed9ca3c92de None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:50.788718 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71292071-a36e-444f-bb2b-aed9ca3c92de None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:50.789379 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71292071-a36e-444f-bb2b-aed9ca3c92de None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:50.790775 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71292071-a36e-444f-bb2b-aed9ca3c92de None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:50.803034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-71292071-a36e-444f-bb2b-aed9ca3c92de None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 22:48:50.803034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:50.803034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:50.803034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:50.803034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:50.803034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:50.803034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:50.803034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:50.803034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 22:48:50.803034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:50.803034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:50.803034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:50.803034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:50.803034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:50.803034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:50.803034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:50.803034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:50.804184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:50.804184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:50.804184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:50.804184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:50.804184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:50.804184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:50.804184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:50.804184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:50.804184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:50.804184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:50.804184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:50.804184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:50.804184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:50.804184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:50.804184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:50.804184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:50.805993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:50.805993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:50.805993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:50.805993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:50.805993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:50.805993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:50.805993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:50.805993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:50.805993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:50.805993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:50.805993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:50.805993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:50.805993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 22:48:50.805993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:50.805993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 472/952] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 22:48:50 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:50.811655 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca24e4b-8580-4e77-b808-a5a835ed2e79 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:50.919626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca24e4b-8580-4e77-b808-a5a835ed2e79 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:50.958742 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b486372-c808-4e92-b98c-e281d93cc67a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:50.959434 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b486372-c808-4e92-b98c-e281d93cc67a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:50.959716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b486372-c808-4e92-b98c-e281d93cc67a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:50.959992 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b486372-c808-4e92-b98c-e281d93cc67a None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:50.960712 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b486372-c808-4e92-b98c-e281d93cc67a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:50.961024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b486372-c808-4e92-b98c-e281d93cc67a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:50.962709 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b486372-c808-4e92-b98c-e281d93cc67a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:50.969249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 481/953] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 22:48:50 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:50.974656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71292071-a36e-444f-bb2b-aed9ca3c92de None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:51.014540 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3627de-a751-4bff-8cdb-1dc24bcd017f None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:51.014540 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3627de-a751-4bff-8cdb-1dc24bcd017f None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:51.014540 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3627de-a751-4bff-8cdb-1dc24bcd017f None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:51.015399 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 482/954] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:48:51.020347 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fd999-1bbd-4ba4-8972-74ad1cc387ea None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:51.020852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fd999-1bbd-4ba4-8972-74ad1cc387ea None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:51.021309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8fd999-1bbd-4ba4-8972-74ad1cc387ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:51.039231 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-cc8fd999-1bbd-4ba4-8972-74ad1cc387ea None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:51.040790 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-cc8fd999-1bbd-4ba4-8972-74ad1cc387ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:51.069425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:51.087690 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71292071-a36e-444f-bb2b-aed9ca3c92de None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:51.122873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 483/955] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:48:51.126254 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794b7b1-6530-4f74-ada5-db69a2c25706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:51.126967 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794b7b1-6530-4f74-ada5-db69a2c25706 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:51.127253 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794b7b1-6530-4f74-ada5-db69a2c25706 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:51.127527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a794b7b1-6530-4f74-ada5-db69a2c25706 None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:51.128769 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794b7b1-6530-4f74-ada5-db69a2c25706 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:51.129081 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794b7b1-6530-4f74-ada5-db69a2c25706 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:51.130437 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a794b7b1-6530-4f74-ada5-db69a2c25706 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:51.136181 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 473/956] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 22:48:50 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:51.147611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794b7b1-6530-4f74-ada5-db69a2c25706 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:51.147753 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fd999-1bbd-4ba4-8972-74ad1cc387ea None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:51.186936 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:51.271202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8fd999-1bbd-4ba4-8972-74ad1cc387ea None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:51.285414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a794b7b1-6530-4f74-ada5-db69a2c25706 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:51.314399 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32ff06-d0e3-44b1-891a-94f6ef898bd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:51.315108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32ff06-d0e3-44b1-891a-94f6ef898bd0 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:51.315381 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32ff06-d0e3-44b1-891a-94f6ef898bd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:51.316192 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe32ff06-d0e3-44b1-891a-94f6ef898bd0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:51.317102 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32ff06-d0e3-44b1-891a-94f6ef898bd0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:51.319151 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe32ff06-d0e3-44b1-891a-94f6ef898bd0 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:51.330486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d531a53-51f5-441f-b10f-01783fce05ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:51.332055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d531a53-51f5-441f-b10f-01783fce05ba None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:51.332389 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d531a53-51f5-441f-b10f-01783fce05ba None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:51.332734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d531a53-51f5-441f-b10f-01783fce05ba None admin] PATH_INFO: `/v3/services/compute` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:51.333500 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d531a53-51f5-441f-b10f-01783fce05ba None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:51.335310 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d531a53-51f5-441f-b10f-01783fce05ba None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:51.342494 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 484/957] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jun 16 22:48:51 2022] POST /identity/v3/endpoints => generated 349 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:48:51.347503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-6d531a53-51f5-441f-b10f-01783fce05ba None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 22:48:51.347503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:51.347503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:51.347503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:51.347503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:51.347503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:51.347503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:51.347503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:51.347503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 22:48:51.347503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:51.347503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:51.347503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:51.347503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:51.347503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:51.347503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:51.347503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:51.347503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:51.348597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:51.348597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:51.348597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:51.348597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:51.348597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:51.348597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:51.348597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:51.348597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:51.348597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:51.348597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:51.348597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:51.348597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:51.348597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:51.348597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:51.348597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:51.348597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:51.349502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:51.349502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:51.349502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:51.349502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:51.349502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:51.349502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:51.349502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:51.349502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:51.349502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:51.349502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:51.349502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:51.349502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:51.349502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 22:48:51.349502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:51.354667 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 474/958] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:51.355154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32ff06-d0e3-44b1-891a-94f6ef898bd0 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:51.377016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:51.478353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe32ff06-d0e3-44b1-891a-94f6ef898bd0 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:51.517633 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a5e629-c0f4-41d3-b0e8-4326bc19ca64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:51.518371 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5e629-c0f4-41d3-b0e8-4326bc19ca64 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:51.519226 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5e629-c0f4-41d3-b0e8-4326bc19ca64 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:51.519464 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a5e629-c0f4-41d3-b0e8-4326bc19ca64 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:51.520171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5e629-c0f4-41d3-b0e8-4326bc19ca64 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:51.520512 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5e629-c0f4-41d3-b0e8-4326bc19ca64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:51.521922 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a5e629-c0f4-41d3-b0e8-4326bc19ca64 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:51.527459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 485/959] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 22:48:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:51.532967 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d531a53-51f5-441f-b10f-01783fce05ba None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:51.632538 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:51.640469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d531a53-51f5-441f-b10f-01783fce05ba None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:51.674197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65586b80-1c62-4f4e-960a-89a1b3898478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:51.675459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65586b80-1c62-4f4e-960a-89a1b3898478 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:51.675744 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65586b80-1c62-4f4e-960a-89a1b3898478 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:51.676019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65586b80-1c62-4f4e-960a-89a1b3898478 None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:51.676719 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65586b80-1c62-4f4e-960a-89a1b3898478 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:51.677025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65586b80-1c62-4f4e-960a-89a1b3898478 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:51.678432 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65586b80-1c62-4f4e-960a-89a1b3898478 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:51.683591 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 475/960] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 22:48:51 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:52.527222 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3b52f-ea56-4449-af48-c64c950ca343 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:52.527712 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3b52f-ea56-4449-af48-c64c950ca343 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:52.527712 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec3b52f-ea56-4449-af48-c64c950ca343 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:52.529296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 486/961] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:52.534784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f525866-a60e-4b23-b2e9-f7ffd6cad077 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:52.535171 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f525866-a60e-4b23-b2e9-f7ffd6cad077 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:52.535352 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f525866-a60e-4b23-b2e9-f7ffd6cad077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:52.552048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-2f525866-a60e-4b23-b2e9-f7ffd6cad077 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:52.553540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-2f525866-a60e-4b23-b2e9-f7ffd6cad077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:52.626708 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 476/962] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:52.645424 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3b52f-ea56-4449-af48-c64c950ca343 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:52.731139 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:52.756137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec3b52f-ea56-4449-af48-c64c950ca343 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:52.772851 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d125e-ce66-4bed-8478-6ac4d534539a None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:52.773140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d125e-ce66-4bed-8478-6ac4d534539a None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:52.773664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74d125e-ce66-4bed-8478-6ac4d534539a None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:52.775083 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 477/963] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:52.780251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d859da9-3cc2-47a8-a5e8-2640752cc564 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:52.780490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d859da9-3cc2-47a8-a5e8-2640752cc564 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:52.780771 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d859da9-3cc2-47a8-a5e8-2640752cc564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:52.793635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1c8a4-775d-4561-a23d-71461acf9ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:52.794723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1c8a4-775d-4561-a23d-71461acf9ed2 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:52.795188 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1c8a4-775d-4561-a23d-71461acf9ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:52.795675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a1c8a4-775d-4561-a23d-71461acf9ed2 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:52.796605 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1c8a4-775d-4561-a23d-71461acf9ed2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:52.798408 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a1c8a4-775d-4561-a23d-71461acf9ed2 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:52.798895 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-5d859da9-3cc2-47a8-a5e8-2640752cc564 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:52.800158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-5d859da9-3cc2-47a8-a5e8-2640752cc564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:52.800436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: WARNING keystone.server.flask.common [None req-25a1c8a4-775d-4561-a23d-71461acf9ed2 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 22:48:52.814540 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.sql.core [None req-25a1c8a4-775d-4561-a23d-71461acf9ed2 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 22:48:52.814540 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 22:48:52.814540 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [parameters: {'user_id': '8ea3de8d348b4c708124837227151d0f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 22:48:52.814540 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80087) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 22:48:52.824437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-25a1c8a4-775d-4561-a23d-71461acf9ed2 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 22:48:52.824437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:52.824437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 22:48:52.824437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 22:48:52.824437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 22:48:52.824437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 22:48:52.824437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 22:48:52.824437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 22:48:52.824437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 22:48:52.824437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application conn.query(q) Jun 16 22:48:52.824437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 22:48:52.824437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 22:48:52.824437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 22:48:52.824437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application result.read() Jun 16 22:48:52.824437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 22:48:52.825302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 22:48:52.825302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 22:48:52.825302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 22:48:52.825302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 22:48:52.825302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 22:48:52.825302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 22:48:52.825302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 22:48:52.825302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 22:48:52.825302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:52.825302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 22:48:52.825302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:52.825302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:52.825302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 22:48:52.825302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 22:48:52.825302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 16 22:48:52.825302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 16 22:48:52.825302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 22:48:52.825302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application next(self.gen) Jun 16 22:48:52.826293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 22:48:52.826293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application yield resource Jun 16 22:48:52.826293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 22:48:52.826293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application next(self.gen) Jun 16 22:48:52.826293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 22:48:52.826293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 22:48:52.826293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 22:48:52.826293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 22:48:52.826293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 22:48:52.826293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise self.value Jun 16 22:48:52.826293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 22:48:52.826293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 22:48:52.826293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 22:48:52.826293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application session.commit() Jun 16 22:48:52.826293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 22:48:52.826293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 22:48:52.826293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 22:48:52.827218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 22:48:52.827218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 22:48:52.827218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self.session.flush() Jun 16 22:48:52.827218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 22:48:52.827218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 22:48:52.827218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 22:48:52.827218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 22:48:52.827218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 22:48:52.827218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application compat.raise_( Jun 16 22:48:52.827218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 22:48:52.827218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception Jun 16 22:48:52.827218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 22:48:52.827218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 22:48:52.827218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 22:48:52.827218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 22:48:52.827218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 22:48:52.827218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 22:48:52.828112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 22:48:52.828112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 22:48:52.828112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 16 22:48:52.828112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 16 22:48:52.828112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 22:48:52.828112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 22:48:52.828112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 22:48:52.828112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 22:48:52.828112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 22:48:52.828112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 22:48:52.828112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 22:48:52.828112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 22:48:52.828112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 22:48:52.828112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 22:48:52.828112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 22:48:52.828112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception Jun 16 22:48:52.828958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 22:48:52.828958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 22:48:52.828958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 22:48:52.828958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 22:48:52.828958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 22:48:52.828958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 22:48:52.828958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 22:48:52.828958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application conn.query(q) Jun 16 22:48:52.828958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 22:48:52.828958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 22:48:52.828958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 22:48:52.828958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application result.read() Jun 16 22:48:52.828958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 22:48:52.828958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 22:48:52.828958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 22:48:52.828958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 22:48:52.829775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 22:48:52.829775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 22:48:52.829775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 22:48:52.829775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 22:48:52.829775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 22:48:52.829775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 22:48:52.829775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [parameters: {'user_id': '8ea3de8d348b4c708124837227151d0f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 22:48:52.829775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 22:48:52.829775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:52.829775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:52.829775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:52.829775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:52.829775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:52.829775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:52.829775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:52.830556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:52.830556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:52.830556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:52.830556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:52.830556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:52.830556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:52.830556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:52.830556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 16 22:48:52.830556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 16 22:48:52.830556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:52.830556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:52.830556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:52.830556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:52.830556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:52.830556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:52.830556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 16 22:48:52.830556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 16 22:48:52.831621 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 16 22:48:52.831621 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 16 22:48:52.831621 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 22:48:52.831621 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 22:48:52.831621 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 22:48:52.831621 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:52.834006 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 487/964] 10.222.0.17 () {60 vars in 1131 bytes} [Thu Jun 16 22:48:52 2022] POST /identity/v3/users => generated 160 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:52.839879 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1c8a4-775d-4561-a23d-71461acf9ed2 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:52.878691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 478/965] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:48:52.902275 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d859da9-3cc2-47a8-a5e8-2640752cc564 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:52.933642 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:52.961417 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a1c8a4-775d-4561-a23d-71461acf9ed2 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:52.999143 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bb398-0f03-40fa-8fe3-8c6eb0bd9102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:53.000331 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bb398-0f03-40fa-8fe3-8c6eb0bd9102 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:53.000848 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bb398-0f03-40fa-8fe3-8c6eb0bd9102 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:53.001325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bb398-0f03-40fa-8fe3-8c6eb0bd9102 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:53.010110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7bb398-0f03-40fa-8fe3-8c6eb0bd9102 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:53.012112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7bb398-0f03-40fa-8fe3-8c6eb0bd9102 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:53.014429 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:53.023538 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d859da9-3cc2-47a8-a5e8-2640752cc564 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:53.030210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-5a7bb398-0f03-40fa-8fe3-8c6eb0bd9102 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 22:48:53.030210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:53.030210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:53.030210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:53.030210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:53.030210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:53.030210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:53.030210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:53.030210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:53.030210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:53.030210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:53.030210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:53.030210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:53.030210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:53.030210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:53.030210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:53.031302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:53.031302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:53.031302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:53.031302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:53.031302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:53.031302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:53.031302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:53.031302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:53.031302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:53.031302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:53.031302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:53.031302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:53.031302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:53.031302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:53.031302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:53.031302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:53.031302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:53.031302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:53.032384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:53.032384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:53.032384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:53.032384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:53.032384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:53.032384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:53.032384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:53.032384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:53.032384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 22:48:53.032384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:53.034677 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 488/966] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:48:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:53.040201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bb398-0f03-40fa-8fe3-8c6eb0bd9102 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:53.069930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ca557-7074-41be-9139-49a7bc9a309f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:53.071435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ca557-7074-41be-9139-49a7bc9a309f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:53.072003 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ca557-7074-41be-9139-49a7bc9a309f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:53.072558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137ca557-7074-41be-9139-49a7bc9a309f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:53.087840 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137ca557-7074-41be-9139-49a7bc9a309f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:48:53.096469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-137ca557-7074-41be-9139-49a7bc9a309f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 22:48:53.096469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:53.096469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:53.096469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:53.096469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:53.096469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:53.096469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:53.096469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:53.096469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:53.096469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:53.096469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:53.096469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:53.096469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:48:53.096469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:48:53.096469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:48:53.096469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:48:53.097492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:48:53.097492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:48:53.097492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:48:53.097492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:48:53.097492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:53.097492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:53.097492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:53.097492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:53.097492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:53.097492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:53.097492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:53.097492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:53.097492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:53.097492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:53.097492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:53.097492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:53.098343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:53.098343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:53.098343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:53.098343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:53.098343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:48:53.098343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:48:53.098343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:48:53.098343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:48:53.098343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:48:53.098343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:48:53.098343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 22:48:53.098343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:53.098343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 479/967] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jun 16 22:48:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:53.105423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ca557-7074-41be-9139-49a7bc9a309f None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:53.165054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bb398-0f03-40fa-8fe3-8c6eb0bd9102 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:53.192232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:53.199958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08c64d-996a-48af-aeb0-227fce9658f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:53.200927 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08c64d-996a-48af-aeb0-227fce9658f3 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:53.201425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08c64d-996a-48af-aeb0-227fce9658f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:53.201917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08c64d-996a-48af-aeb0-227fce9658f3 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:53.202965 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08c64d-996a-48af-aeb0-227fce9658f3 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:53.203497 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08c64d-996a-48af-aeb0-227fce9658f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:53.205114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08c64d-996a-48af-aeb0-227fce9658f3 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:53.220832 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137ca557-7074-41be-9139-49a7bc9a309f None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:53.246785 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 489/968] 10.222.0.17 () {56 vars in 1101 bytes} [Thu Jun 16 22:48:53 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:53.269919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ba79d-e764-4aee-95c9-d6da4c36b323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:53.270637 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ba79d-e764-4aee-95c9-d6da4c36b323 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:53.270983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ba79d-e764-4aee-95c9-d6da4c36b323 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:53.271267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6ba79d-e764-4aee-95c9-d6da4c36b323 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:53.272032 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ba79d-e764-4aee-95c9-d6da4c36b323 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:53.272343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ba79d-e764-4aee-95c9-d6da4c36b323 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:53.273744 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6ba79d-e764-4aee-95c9-d6da4c36b323 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:53.284916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 480/969] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jun 16 22:48:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:53.290557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08c64d-996a-48af-aeb0-227fce9658f3 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:53.356420 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:53.404903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08c64d-996a-48af-aeb0-227fce9658f3 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:53.443273 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5661d6-478d-4421-91d4-6c2be9ea1bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:53.444084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5661d6-478d-4421-91d4-6c2be9ea1bf4 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:53.444368 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5661d6-478d-4421-91d4-6c2be9ea1bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:53.444644 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5661d6-478d-4421-91d4-6c2be9ea1bf4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:53.455322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5661d6-478d-4421-91d4-6c2be9ea1bf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:53.456930 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5661d6-478d-4421-91d4-6c2be9ea1bf4 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:53.476400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-3b5661d6-478d-4421-91d4-6c2be9ea1bf4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:53.476400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:53.476400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:53.476400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:53.476400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:53.476400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:53.476400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:53.476400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:53.476400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:53.476400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:53.476400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:53.476400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:53.476400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:53.476400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:53.476400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:53.478518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:53.478518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:53.478518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:53.478518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:53.478518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:53.478518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:53.478518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:53.478518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:53.478518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:53.478518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:53.478518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:53.478518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:53.478518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:53.478518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:53.478518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:53.478518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:53.478518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:53.479931 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:53.479931 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:53.479931 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:53.479931 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:53.479931 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 490/970] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:53.483638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ba79d-e764-4aee-95c9-d6da4c36b323 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:53.594441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6ba79d-e764-4aee-95c9-d6da4c36b323 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:53.608580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7917e8a-8aa2-4ad4-a83b-005d1ae6ff84 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:53.609058 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7917e8a-8aa2-4ad4-a83b-005d1ae6ff84 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:53.609165 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7917e8a-8aa2-4ad4-a83b-005d1ae6ff84 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:53.610633 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 491/971] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:53.616116 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4752d-305a-43ab-ba6d-94d23ce63689 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:53.616405 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4752d-305a-43ab-ba6d-94d23ce63689 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:53.616684 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd4752d-305a-43ab-ba6d-94d23ce63689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:53.631224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb4c91-0bdd-4eb1-bc64-8439df12f3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:53.631892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb4c91-0bdd-4eb1-bc64-8439df12f3f0 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:53.632177 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb4c91-0bdd-4eb1-bc64-8439df12f3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:53.632455 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfb4c91-0bdd-4eb1-bc64-8439df12f3f0 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:53.633176 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb4c91-0bdd-4eb1-bc64-8439df12f3f0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:53.633483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb4c91-0bdd-4eb1-bc64-8439df12f3f0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:53.634242 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-efd4752d-305a-43ab-ba6d-94d23ce63689 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:53.634876 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfb4c91-0bdd-4eb1-bc64-8439df12f3f0 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:53.635846 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-efd4752d-305a-43ab-ba6d-94d23ce63689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:53.644073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:53.650748 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 481/972] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:53.654633 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb4c91-0bdd-4eb1-bc64-8439df12f3f0 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:53.710351 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 492/973] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:53.733023 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4752d-305a-43ab-ba6d-94d23ce63689 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:53.767564 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfb4c91-0bdd-4eb1-bc64-8439df12f3f0 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:53.778192 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:53.808852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87ebfc-3652-4448-a1e1-d2b52c50f81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:53.809692 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87ebfc-3652-4448-a1e1-d2b52c50f81d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:53.809980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87ebfc-3652-4448-a1e1-d2b52c50f81d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:53.810260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f87ebfc-3652-4448-a1e1-d2b52c50f81d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:53.820120 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87ebfc-3652-4448-a1e1-d2b52c50f81d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:53.822057 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f87ebfc-3652-4448-a1e1-d2b52c50f81d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:53.839293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-7f87ebfc-3652-4448-a1e1-d2b52c50f81d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 22:48:53.839293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:53.839293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:53.839293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:53.839293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:53.839293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:53.839293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:53.839293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:53.839293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:53.839293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:53.839293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:53.839293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:53.839293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:48:53.839293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:48:53.839293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:48:53.839293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:48:53.840318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:53.840318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:53.840318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:48:53.840318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:53.840318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:48:53.840318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:48:53.840318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:53.840318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:53.840318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:53.840318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:53.840318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:53.840318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:53.840318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:53.840318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:53.840318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:53.840318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:53.840318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:53.840318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:53.841271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:53.841271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:53.841271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:48:53.841271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:48:53.841271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:48:53.841271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:48:53.841271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:48:53.841271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:48:53.841271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 22:48:53.841271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:53.841271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 482/974] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 22:48:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:53.846373 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87ebfc-3652-4448-a1e1-d2b52c50f81d None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:53.849305 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd4752d-305a-43ab-ba6d-94d23ce63689 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:53.890365 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48bc31d4-7bd8-4bc1-b8cb-89fa7b2ee772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:53.891211 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc31d4-7bd8-4bc1-b8cb-89fa7b2ee772 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:53.891495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc31d4-7bd8-4bc1-b8cb-89fa7b2ee772 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:53.891774 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48bc31d4-7bd8-4bc1-b8cb-89fa7b2ee772 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:53.892432 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc31d4-7bd8-4bc1-b8cb-89fa7b2ee772 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:53.894315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48bc31d4-7bd8-4bc1-b8cb-89fa7b2ee772 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:53.907026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-48bc31d4-7bd8-4bc1-b8cb-89fa7b2ee772 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 22:48:53.907026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:53.907026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:53.907026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:53.907026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:53.907026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:53.907026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:53.907026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:53.907026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 22:48:53.907026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:53.907026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:53.907026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:53.907026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:53.907026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:53.907026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:53.907026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:53.907026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:53.908084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:53.908084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:53.908084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:53.908084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:53.908084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:53.908084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:53.908084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:53.908084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:53.908084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:53.908084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:53.908084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:53.908084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:53.908084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:53.908084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:53.908084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:53.908084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:53.908903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:53.908903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:53.908903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:53.908903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:53.908903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:53.908903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:53.908903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:53.908903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:53.908903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:53.908903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:53.908903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:53.908903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:53.908903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 22:48:53.908903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:53.908903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 493/975] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 22:48:53 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:53.910639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b9f98-3b4b-4b74-8269-856b9ab170df None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:53.910639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b9f98-3b4b-4b74-8269-856b9ab170df None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:53.910639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b9f98-3b4b-4b74-8269-856b9ab170df None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:53.911812 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 494/976] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:53.914509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b9f98-3b4b-4b74-8269-856b9ab170df None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:53.918130 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:53.958764 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f87ebfc-3652-4448-a1e1-d2b52c50f81d None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:53.993049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6812505a-a796-44e1-a2e1-bafc5296a9ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:53.993731 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812505a-a796-44e1-a2e1-bafc5296a9ba None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:53.994020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812505a-a796-44e1-a2e1-bafc5296a9ba None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:53.994346 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6812505a-a796-44e1-a2e1-bafc5296a9ba None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:53.995229 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812505a-a796-44e1-a2e1-bafc5296a9ba None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:53.995546 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812505a-a796-44e1-a2e1-bafc5296a9ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:53.996975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6812505a-a796-44e1-a2e1-bafc5296a9ba None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:54.014800 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 483/977] 10.222.0.17 () {56 vars in 1149 bytes} [Thu Jun 16 22:48:53 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:54.016007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c1206-e367-41bb-b84c-856148e27451 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:54.016297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c1206-e367-41bb-b84c-856148e27451 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:54.016579 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331c1206-e367-41bb-b84c-856148e27451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:54.022134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b9f98-3b4b-4b74-8269-856b9ab170df None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:54.023723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:54.033118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-331c1206-e367-41bb-b84c-856148e27451 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:54.034349 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-331c1206-e367-41bb-b84c-856148e27451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:54.055918 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90de3c2-d060-421f-9653-6c2bbac7d74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:54.056586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90de3c2-d060-421f-9653-6c2bbac7d74a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:54.056867 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90de3c2-d060-421f-9653-6c2bbac7d74a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:54.057144 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90de3c2-d060-421f-9653-6c2bbac7d74a None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:54.057843 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90de3c2-d060-421f-9653-6c2bbac7d74a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:54.058149 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90de3c2-d060-421f-9653-6c2bbac7d74a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:54.059520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90de3c2-d060-421f-9653-6c2bbac7d74a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:54.064290 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 495/978] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 22:48:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:54.069404 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90de3c2-d060-421f-9653-6c2bbac7d74a None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:54.101842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 484/979] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:54.102584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c1206-e367-41bb-b84c-856148e27451 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:54.150964 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:54.167957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90de3c2-d060-421f-9653-6c2bbac7d74a None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:54.202850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38644973-7efe-46c3-94a1-2c5d35b4aebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:54.203503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38644973-7efe-46c3-94a1-2c5d35b4aebd None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:54.203790 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38644973-7efe-46c3-94a1-2c5d35b4aebd None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:54.204071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38644973-7efe-46c3-94a1-2c5d35b4aebd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:54.207883 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331c1206-e367-41bb-b84c-856148e27451 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:54.240652 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:54.244365 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1eee92-4276-4873-bdf3-1cca2a9158cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:54.245035 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1eee92-4276-4873-bdf3-1cca2a9158cb None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:54.245322 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1eee92-4276-4873-bdf3-1cca2a9158cb None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:54.245603 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1eee92-4276-4873-bdf3-1cca2a9158cb None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:54.246666 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1eee92-4276-4873-bdf3-1cca2a9158cb None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:54.246914 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1eee92-4276-4873-bdf3-1cca2a9158cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:54.248294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1eee92-4276-4873-bdf3-1cca2a9158cb None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:54.253759 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 485/980] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 22:48:54 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:54.254418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1eee92-4276-4873-bdf3-1cca2a9158cb None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:54.276030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38644973-7efe-46c3-94a1-2c5d35b4aebd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:54.278131 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38644973-7efe-46c3-94a1-2c5d35b4aebd None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:54.352368 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1eee92-4276-4873-bdf3-1cca2a9158cb None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:54.364895 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:54.374970 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 496/981] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:48:54 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:48:54.376034 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38644973-7efe-46c3-94a1-2c5d35b4aebd None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:54.387632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2087f1-9559-4d71-8c30-c68650b97ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:54.388301 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2087f1-9559-4d71-8c30-c68650b97ed6 None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:54.388585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2087f1-9559-4d71-8c30-c68650b97ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:54.388863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2087f1-9559-4d71-8c30-c68650b97ed6 None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:54.389573 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2087f1-9559-4d71-8c30-c68650b97ed6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:54.390968 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2087f1-9559-4d71-8c30-c68650b97ed6 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:54.404908 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 486/982] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 22:48:54 2022] POST /identity/v3/services => generated 242 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:48:54.405617 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2087f1-9559-4d71-8c30-c68650b97ed6 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:54.448598 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:54.489024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38644973-7efe-46c3-94a1-2c5d35b4aebd None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:54.532985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2087f1-9559-4d71-8c30-c68650b97ed6 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:54.535388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fde42c-9dfe-4b93-b54c-d9906e256f45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:54.536069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fde42c-9dfe-4b93-b54c-d9906e256f45 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:54.536361 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fde42c-9dfe-4b93-b54c-d9906e256f45 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:54.536638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fde42c-9dfe-4b93-b54c-d9906e256f45 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:54.537451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fde42c-9dfe-4b93-b54c-d9906e256f45 None admin] RBAC: Adding query filter params (interface=admin, service_id=85a71b72ad414bfd90c542849583a722, region_id=RegionOne) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:54.537763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fde42c-9dfe-4b93-b54c-d9906e256f45 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:54.539187 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fde42c-9dfe-4b93-b54c-d9906e256f45 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:54.545125 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 497/983] 10.222.0.17 () {56 vars in 1249 bytes} [Thu Jun 16 22:48:54 2022] GET /identity/v3/endpoints?service_id=85a71b72ad414bfd90c542849583a722&interface=admin®ion_id=RegionOne => generated 193 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:54.551515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fde42c-9dfe-4b93-b54c-d9906e256f45 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:54.582053 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb136604-ca83-4984-a457-51e2688411ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:54.582515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb136604-ca83-4984-a457-51e2688411ba None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:54.582856 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb136604-ca83-4984-a457-51e2688411ba None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:54.583137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb136604-ca83-4984-a457-51e2688411ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:54.595663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb136604-ca83-4984-a457-51e2688411ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:54.601179 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb136604-ca83-4984-a457-51e2688411ba None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:54.616406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-bb136604-ca83-4984-a457-51e2688411ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:54.616406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:54.616406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:54.616406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:54.616406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:54.616406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:54.616406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:54.616406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:54.616406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:54.616406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:54.616406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:54.616406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:54.616406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:48:54.616406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:48:54.616406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:48:54.617561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:48:54.617561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:54.617561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:54.617561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:54.617561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:54.617561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:54.617561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:54.617561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:54.617561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:54.617561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:54.617561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:54.617561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:54.617561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:54.617561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:54.617561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:54.617561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:54.617561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:54.618846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:48:54.618846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:48:54.618846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:48:54.618846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:54.618846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 487/984] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:54.623441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb136604-ca83-4984-a457-51e2688411ba None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:54.646568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:54.670160 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fde42c-9dfe-4b93-b54c-d9906e256f45 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:54.697610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:54.706977 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61824bfe-a8ea-4619-88f6-5196fbe60909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:54.707755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61824bfe-a8ea-4619-88f6-5196fbe60909 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:54.708043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61824bfe-a8ea-4619-88f6-5196fbe60909 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:54.708320 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61824bfe-a8ea-4619-88f6-5196fbe60909 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:54.708998 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61824bfe-a8ea-4619-88f6-5196fbe60909 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:54.710449 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61824bfe-a8ea-4619-88f6-5196fbe60909 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:54.726919 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 498/985] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jun 16 22:48:54 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 22:48:54.745514 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb136604-ca83-4984-a457-51e2688411ba None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:54.784631 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd7b76b-5ef2-468e-bf8f-9981e41cb238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:54.785320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7b76b-5ef2-468e-bf8f-9981e41cb238 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:54.785610 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7b76b-5ef2-468e-bf8f-9981e41cb238 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:54.785894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd7b76b-5ef2-468e-bf8f-9981e41cb238 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:54.786676 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd7b76b-5ef2-468e-bf8f-9981e41cb238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:54.786995 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd7b76b-5ef2-468e-bf8f-9981e41cb238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:54.788420 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd7b76b-5ef2-468e-bf8f-9981e41cb238 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:54.802536 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 488/986] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:48:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:54.808143 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61824bfe-a8ea-4619-88f6-5196fbe60909 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:54.918885 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61824bfe-a8ea-4619-88f6-5196fbe60909 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:54.953674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5f457-a50b-4ede-b99b-5ced01fbea2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:54.954477 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5f457-a50b-4ede-b99b-5ced01fbea2e None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:54.954792 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5f457-a50b-4ede-b99b-5ced01fbea2e None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:54.955070 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef5f457-a50b-4ede-b99b-5ced01fbea2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:54.962303 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5f457-a50b-4ede-b99b-5ced01fbea2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:54.963929 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef5f457-a50b-4ede-b99b-5ced01fbea2e None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:54.978419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-5ef5f457-a50b-4ede-b99b-5ced01fbea2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:54.978419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:54.978419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:54.978419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:54.978419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:54.978419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:54.978419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:54.978419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:54.978419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:54.978419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:54.978419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:54.978419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:54.978419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:48:54.978419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:48:54.978419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:48:54.979548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:48:54.979548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:54.979548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:54.979548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:54.979548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:54.979548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:54.979548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:54.979548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:54.979548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:54.979548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:54.979548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:54.979548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:54.979548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:54.979548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:54.979548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:54.979548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:54.979548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:54.985137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:48:54.985137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:48:54.985137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:48:54.985137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:48:54.985137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:48:54.985137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:48:54.985137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:48:54.985137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:54.985137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 499/987] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:54.987767 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd7b76b-5ef2-468e-bf8f-9981e41cb238 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:55.071394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:55.091434 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd7b76b-5ef2-468e-bf8f-9981e41cb238 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:55.125738 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fec02-b31a-4031-94ed-2c8f4936eb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:55.126401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fec02-b31a-4031-94ed-2c8f4936eb06 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:55.126716 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fec02-b31a-4031-94ed-2c8f4936eb06 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:55.127001 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fec02-b31a-4031-94ed-2c8f4936eb06 None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:55.127798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fec02-b31a-4031-94ed-2c8f4936eb06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:55.128119 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fec02-b31a-4031-94ed-2c8f4936eb06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:55.129534 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15fec02-b31a-4031-94ed-2c8f4936eb06 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:55.146360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 489/988] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:48:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:55.147607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5f457-a50b-4ede-b99b-5ced01fbea2e None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:55.199590 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:55.259181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef5f457-a50b-4ede-b99b-5ced01fbea2e None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:55.297987 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be239a41-08f7-44db-8b68-c0de49e8afb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:55.298674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be239a41-08f7-44db-8b68-c0de49e8afb0 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:55.298967 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be239a41-08f7-44db-8b68-c0de49e8afb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:55.299248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be239a41-08f7-44db-8b68-c0de49e8afb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:55.300601 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be239a41-08f7-44db-8b68-c0de49e8afb0 None admin] RBAC: Adding query filter params (role.id=c6bedc2399d04c78a942f96283ed4560, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=faf778e713b54a638c1db4a5db6180b6) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:55.300916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be239a41-08f7-44db-8b68-c0de49e8afb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:55.302339 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be239a41-08f7-44db-8b68-c0de49e8afb0 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:55.308542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 500/989] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:48:55 2022] GET /identity/v3/role_assignments?user.id=faf778e713b54a638c1db4a5db6180b6&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=c6bedc2399d04c78a942f96283ed4560 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:56.579782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5bf4e-1e14-4238-9e31-c2fe9d283c78 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:56.580266 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5bf4e-1e14-4238-9e31-c2fe9d283c78 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:56.580266 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d5bf4e-1e14-4238-9e31-c2fe9d283c78 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:56.582216 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 490/990] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:56.587542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26449d1a-8598-4778-a2b4-e694e8a47e04 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:56.587952 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26449d1a-8598-4778-a2b4-e694e8a47e04 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:56.588108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26449d1a-8598-4778-a2b4-e694e8a47e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:56.604737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-26449d1a-8598-4778-a2b4-e694e8a47e04 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:56.605899 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-26449d1a-8598-4778-a2b4-e694e8a47e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:56.674861 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 501/991] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:56.696674 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5bf4e-1e14-4238-9e31-c2fe9d283c78 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:56.801057 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d5bf4e-1e14-4238-9e31-c2fe9d283c78 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:56.838067 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a766dbb8-4a77-4197-8792-8c1c402b2b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:56.838816 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766dbb8-4a77-4197-8792-8c1c402b2b2f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:56.839102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766dbb8-4a77-4197-8792-8c1c402b2b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:56.839434 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a766dbb8-4a77-4197-8792-8c1c402b2b2f None admin] PATH_INFO: `/v3/services/compute` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:56.840024 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766dbb8-4a77-4197-8792-8c1c402b2b2f None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:56.841382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a766dbb8-4a77-4197-8792-8c1c402b2b2f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:56.853529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-a766dbb8-4a77-4197-8792-8c1c402b2b2f None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 22:48:56.853529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:56.853529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:56.853529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:56.853529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:56.853529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:56.853529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:56.853529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:56.853529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 22:48:56.853529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:56.853529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:56.853529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:56.853529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:56.853529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:56.853529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:56.853529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:56.853529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:56.854507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:56.854507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:56.854507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:56.854507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:56.854507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:56.854507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:56.854507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:56.854507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:56.854507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:56.854507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:56.854507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:56.854507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:56.854507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:56.854507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:56.854507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:56.854507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:56.855710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:56.855710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:56.855710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:56.855710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:56.855710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:56.855710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:56.855710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:56.855710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:56.855710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:56.855710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:56.855710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:56.855710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:56.855710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 22:48:56.855710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:56.858676 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 491/992] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:56 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:56.860916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26449d1a-8598-4778-a2b4-e694e8a47e04 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:56.866891 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06ec32-36db-4e6f-8f52-c7ba7ec11cd0 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:56.867188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06ec32-36db-4e6f-8f52-c7ba7ec11cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:56.867474 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf06ec32-36db-4e6f-8f52-c7ba7ec11cd0 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:56.868948 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 492/993] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:48:56.873790 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f75657-3073-4fde-99b6-820ddf07bf26 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:56.874080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f75657-3073-4fde-99b6-820ddf07bf26 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:56.874358 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f75657-3073-4fde-99b6-820ddf07bf26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:56.882302 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:56.893953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-19f75657-3073-4fde-99b6-820ddf07bf26 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:56.895168 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-19f75657-3073-4fde-99b6-820ddf07bf26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:56.915950 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:56.966680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 493/994] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:56 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:48:56.974068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26449d1a-8598-4778-a2b4-e694e8a47e04 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:56.991278 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f75657-3073-4fde-99b6-820ddf07bf26 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:57.010580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71224061-b3c4-4e93-a13b-000edaf5a53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:57.011278 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71224061-b3c4-4e93-a13b-000edaf5a53d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:57.011563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71224061-b3c4-4e93-a13b-000edaf5a53d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:57.011840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71224061-b3c4-4e93-a13b-000edaf5a53d None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:57.012549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71224061-b3c4-4e93-a13b-000edaf5a53d None admin] RBAC: Adding query filter params (name=compute) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:57.012862 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71224061-b3c4-4e93-a13b-000edaf5a53d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:57.014215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71224061-b3c4-4e93-a13b-000edaf5a53d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:57.023567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 502/995] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 22:48:56 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:57.024613 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71224061-b3c4-4e93-a13b-000edaf5a53d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:57.114029 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f75657-3073-4fde-99b6-820ddf07bf26 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:57.128067 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71224061-b3c4-4e93-a13b-000edaf5a53d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:57.138110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:57.155445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf0186-81b9-4f4b-8a21-31c0806909d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:57.156831 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf0186-81b9-4f4b-8a21-31c0806909d5 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:57.157686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf0186-81b9-4f4b-8a21-31c0806909d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:57.158381 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cf0186-81b9-4f4b-8a21-31c0806909d5 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:57.160262 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf0186-81b9-4f4b-8a21-31c0806909d5 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:57.161830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cf0186-81b9-4f4b-8a21-31c0806909d5 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:57.163822 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:57.165788 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed6b59-5430-4bd8-b678-c64663973bb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:57.166513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed6b59-5430-4bd8-b678-c64663973bb6 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:57.166783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed6b59-5430-4bd8-b678-c64663973bb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:57.167067 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed6b59-5430-4bd8-b678-c64663973bb6 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:57.167781 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed6b59-5430-4bd8-b678-c64663973bb6 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:57.168094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed6b59-5430-4bd8-b678-c64663973bb6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:57.169451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ed6b59-5430-4bd8-b678-c64663973bb6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:57.179846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-18cf0186-81b9-4f4b-8a21-31c0806909d5 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 22:48:57.179846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:57.179846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:57.179846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:57.179846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:57.179846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:57.179846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:57.179846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:57.179846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 22:48:57.179846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:57.179846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:57.179846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:57.179846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:57.179846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:57.179846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:57.179846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:57.179846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:57.180871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:57.180871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:57.180871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:57.180871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:57.180871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:57.180871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:57.180871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:57.180871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:57.180871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:57.180871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:57.180871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:57.180871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:57.180871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:57.180871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:57.180871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:57.180871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:57.181817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:57.181817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:57.181817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:57.181817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:57.181817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:57.181817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:57.181817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:57.181817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:57.181817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:57.181817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:57.181817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:57.181817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:57.181817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 22:48:57.181817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:57.182549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 503/996] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 22:48:57 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:57.182652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 494/997] 10.222.0.17 () {56 vars in 1126 bytes} [Thu Jun 16 22:48:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:57.182724 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed6b59-5430-4bd8-b678-c64663973bb6 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:57.182805 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf0186-81b9-4f4b-8a21-31c0806909d5 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:57.292009 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed6b59-5430-4bd8-b678-c64663973bb6 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:57.295441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cf0186-81b9-4f4b-8a21-31c0806909d5 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:57.332191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2ec99-0bc9-49ff-9eed-e236bd6c1993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:57.332865 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2ec99-0bc9-49ff-9eed-e236bd6c1993 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:57.333174 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2ec99-0bc9-49ff-9eed-e236bd6c1993 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:57.333441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec2ec99-0bc9-49ff-9eed-e236bd6c1993 None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:57.334136 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2ec99-0bc9-49ff-9eed-e236bd6c1993 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:57.334452 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2ec99-0bc9-49ff-9eed-e236bd6c1993 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:57.335536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f6aa7-d5d0-4631-841d-06df6f45036d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:57.335866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec2ec99-0bc9-49ff-9eed-e236bd6c1993 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:57.336591 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f6aa7-d5d0-4631-841d-06df6f45036d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:57.337125 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f6aa7-d5d0-4631-841d-06df6f45036d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:57.337703 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564f6aa7-d5d0-4631-841d-06df6f45036d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:57.338500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f6aa7-d5d0-4631-841d-06df6f45036d None admin] RBAC: Adding query filter params (interface=public, service_id=856c7690f0344d4ebb1ab249471f1eb6, region_id=RegionOne) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:57.338849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f6aa7-d5d0-4631-841d-06df6f45036d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:57.340190 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564f6aa7-d5d0-4631-841d-06df6f45036d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:57.345718 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 495/998] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 22:48:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:57.347177 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2ec99-0bc9-49ff-9eed-e236bd6c1993 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:57.350966 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 504/999] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jun 16 22:48:57 2022] GET /identity/v3/endpoints?service_id=856c7690f0344d4ebb1ab249471f1eb6&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:48:57.360364 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f6aa7-d5d0-4631-841d-06df6f45036d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:57.411567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:57.423222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:57.464365 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec2ec99-0bc9-49ff-9eed-e236bd6c1993 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:57.481866 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564f6aa7-d5d0-4631-841d-06df6f45036d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:57.497602 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40886b46-a260-4b9e-b8e5-9de8997d1826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:57.498271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40886b46-a260-4b9e-b8e5-9de8997d1826 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:57.498559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40886b46-a260-4b9e-b8e5-9de8997d1826 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:57.498873 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40886b46-a260-4b9e-b8e5-9de8997d1826 None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:57.499583 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40886b46-a260-4b9e-b8e5-9de8997d1826 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:57.499892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40886b46-a260-4b9e-b8e5-9de8997d1826 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:57.501263 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40886b46-a260-4b9e-b8e5-9de8997d1826 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:57.506043 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 496/1000] 10.222.0.17 () {56 vars in 1122 bytes} [Thu Jun 16 22:48:57 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:57.507260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6b2d1-0d46-4f94-ad42-003ee5297054 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:57.507550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6b2d1-0d46-4f94-ad42-003ee5297054 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:57.507830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed6b2d1-0d46-4f94-ad42-003ee5297054 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:57.509067 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 497/1001] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:57.511393 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6b2d1-0d46-4f94-ad42-003ee5297054 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:57.521559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ec2e65-5f1a-4f4b-bedd-fdb9cc2d2016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:57.522569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec2e65-5f1a-4f4b-bedd-fdb9cc2d2016 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:57.522900 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec2e65-5f1a-4f4b-bedd-fdb9cc2d2016 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:57.523185 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ec2e65-5f1a-4f4b-bedd-fdb9cc2d2016 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:57.524183 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec2e65-5f1a-4f4b-bedd-fdb9cc2d2016 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:57.525546 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ec2e65-5f1a-4f4b-bedd-fdb9cc2d2016 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:57.546625 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 505/1002] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jun 16 22:48:57 2022] GET /identity/v3/services => generated 1727 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 22:48:57.546625 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03c41d-fcbc-4e8a-9813-9233d04f85cb None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:57.546625 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03c41d-fcbc-4e8a-9813-9233d04f85cb None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:57.546625 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b03c41d-fcbc-4e8a-9813-9233d04f85cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:57.551565 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-5b03c41d-fcbc-4e8a-9813-9233d04f85cb None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:57.552793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5b03c41d-fcbc-4e8a-9813-9233d04f85cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:57.625954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 506/1003] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:57 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:57.626662 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed6b2d1-0d46-4f94-ad42-003ee5297054 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:57.648788 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03c41d-fcbc-4e8a-9813-9233d04f85cb None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:57.655327 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:57.668120 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8308bce-d925-48a4-ad61-176307f0da7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:57.668814 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8308bce-d925-48a4-ad61-176307f0da7b None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:57.669096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8308bce-d925-48a4-ad61-176307f0da7b None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:57.669371 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8308bce-d925-48a4-ad61-176307f0da7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:57.670127 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8308bce-d925-48a4-ad61-176307f0da7b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:57.671673 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8308bce-d925-48a4-ad61-176307f0da7b None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:57.681072 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:57.698819 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 498/1004] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jun 16 22:48:57 2022] POST /identity/v3/endpoints => generated 325 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 22:48:57.765225 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b03c41d-fcbc-4e8a-9813-9233d04f85cb None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:57.811341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b518876-eb2f-4a31-942b-8474c2f2f4be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:57.812208 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b518876-eb2f-4a31-942b-8474c2f2f4be None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:57.812499 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b518876-eb2f-4a31-942b-8474c2f2f4be None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:57.812776 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b518876-eb2f-4a31-942b-8474c2f2f4be None admin] PATH_INFO: `/v3/services/image` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:57.813441 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b518876-eb2f-4a31-942b-8474c2f2f4be None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:57.814911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b518876-eb2f-4a31-942b-8474c2f2f4be None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:57.827738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-2b518876-eb2f-4a31-942b-8474c2f2f4be None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 22:48:57.827738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:57.827738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:57.827738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:57.827738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:57.827738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:57.827738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:57.827738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:57.827738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 22:48:57.827738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:57.827738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:57.827738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:57.827738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:57.827738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:57.827738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:57.827738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:57.827738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:57.828791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:57.828791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:57.828791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:57.828791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:57.828791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:57.828791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:57.828791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:57.828791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:57.828791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:57.828791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:57.828791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:57.828791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:57.828791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:57.828791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:57.828791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:57.828791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:57.829637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:57.829637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:57.829637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:57.829637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:57.829637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:57.829637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:57.829637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:57.829637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:57.829637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:57.829637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:57.829637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:57.829637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:57.829637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 22:48:57.829637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:48:57.831017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 507/1005] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jun 16 22:48:57 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:48:57.835563 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8308bce-d925-48a4-ad61-176307f0da7b None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:57.943553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8308bce-d925-48a4-ad61-176307f0da7b None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:57.978187 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a672814d-5fd7-4a72-970b-8e5fccc2c51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:57.978896 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672814d-5fd7-4a72-970b-8e5fccc2c51c None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:57.979188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672814d-5fd7-4a72-970b-8e5fccc2c51c None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:57.979470 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a672814d-5fd7-4a72-970b-8e5fccc2c51c None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:57.980271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672814d-5fd7-4a72-970b-8e5fccc2c51c None admin] RBAC: Adding query filter params (name=image) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:57.980586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672814d-5fd7-4a72-970b-8e5fccc2c51c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:57.982070 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a672814d-5fd7-4a72-970b-8e5fccc2c51c None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:57.987973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 499/1006] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:57 2022] GET /identity/v3/services?name=image => generated 122 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:57.993713 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b518876-eb2f-4a31-942b-8474c2f2f4be None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:58.090006 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:58.101081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b518876-eb2f-4a31-942b-8474c2f2f4be None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:58.135274 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a63928-00b7-45e4-bd02-779cfa1bd601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:58.135948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a63928-00b7-45e4-bd02-779cfa1bd601 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:58.136237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a63928-00b7-45e4-bd02-779cfa1bd601 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:58.136518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a63928-00b7-45e4-bd02-779cfa1bd601 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:58.137227 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a63928-00b7-45e4-bd02-779cfa1bd601 None admin] RBAC: Adding query filter params (type=image) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:48:58.137537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a63928-00b7-45e4-bd02-779cfa1bd601 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:58.138933 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a63928-00b7-45e4-bd02-779cfa1bd601 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:58.150631 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 508/1007] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:57 2022] GET /identity/v3/services?type=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:48:59.692198 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f38845-6ae2-4020-ac3a-af8f33a71493 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:59.692718 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f38845-6ae2-4020-ac3a-af8f33a71493 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:59.692796 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f38845-6ae2-4020-ac3a-af8f33a71493 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:59.694394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 500/1008] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:59.699321 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e41852-abca-4ba5-ad13-450a542c91d7 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:59.699720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e41852-abca-4ba5-ad13-450a542c91d7 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:59.699896 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e41852-abca-4ba5-ad13-450a542c91d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:59.716941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-36e41852-abca-4ba5-ad13-450a542c91d7 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:59.718114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-36e41852-abca-4ba5-ad13-450a542c91d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:59.790325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 509/1009] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:59 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:48:59.812650 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f38845-6ae2-4020-ac3a-af8f33a71493 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:59.833439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb03b9-4e8b-4e9d-ac94-49ab61844dcb None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:59.833744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb03b9-4e8b-4e9d-ac94-49ab61844dcb None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:59.834006 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadb03b9-4e8b-4e9d-ac94-49ab61844dcb None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:59.835542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 510/1010] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:48:59.840315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542d357-e070-4da2-aa52-c738188fd62e None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:59.840608 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542d357-e070-4da2-aa52-c738188fd62e None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:59.840892 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542d357-e070-4da2-aa52-c738188fd62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:59.858181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-9542d357-e070-4da2-aa52-c738188fd62e None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:48:59.859386 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-9542d357-e070-4da2-aa52-c738188fd62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:48:59.866137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:59.928409 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f38845-6ae2-4020-ac3a-af8f33a71493 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:48:59.932781 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 511/1011] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:48:59 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:48:59.956014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542d357-e070-4da2-aa52-c738188fd62e None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:48:59.964916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6ce71-86e2-4ee5-84d2-aaae04e4bb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:48:59.965719 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6ce71-86e2-4ee5-84d2-aaae04e4bb01 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:48:59.966002 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6ce71-86e2-4ee5-84d2-aaae04e4bb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:48:59.966277 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f6ce71-86e2-4ee5-84d2-aaae04e4bb01 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:48:59.966956 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6ce71-86e2-4ee5-84d2-aaae04e4bb01 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:48:59.968310 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f6ce71-86e2-4ee5-84d2-aaae04e4bb01 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:48:59.971364 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:48:59.980331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-a0f6ce71-86e2-4ee5-84d2-aaae04e4bb01 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 22:48:59.980331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:59.980331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:48:59.980331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:48:59.980331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:48:59.980331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:48:59.980331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:48:59.980331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:59.980331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 22:48:59.980331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:59.980331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:48:59.980331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:59.980331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:48:59.980331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:48:59.980331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:48:59.980331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:48:59.980331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:48:59.981294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:48:59.981294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:48:59.981294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:48:59.981294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:48:59.981294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:48:59.981294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:48:59.981294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:48:59.981294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:48:59.981294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:48:59.981294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:48:59.981294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:48:59.981294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:48:59.981294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:48:59.981294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:48:59.981294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:48:59.981294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:48:59.982118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:48:59.982118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:48:59.982118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:48:59.982118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:48:59.982118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:48:59.982118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:48:59.982118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:48:59.982118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:48:59.982118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:48:59.982118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:48:59.982118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:48:59.982118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:48:59.982118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 22:48:59.982118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:48:59.982118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 501/1012] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:48:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:48:59.987854 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6ce71-86e2-4ee5-84d2-aaae04e4bb01 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:00.065902 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542d357-e070-4da2-aa52-c738188fd62e None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:00.093236 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f6ce71-86e2-4ee5-84d2-aaae04e4bb01 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:00.102854 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69955e7f-c79f-4f5a-b199-4584ad501dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:00.103660 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69955e7f-c79f-4f5a-b199-4584ad501dc3 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:00.103939 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69955e7f-c79f-4f5a-b199-4584ad501dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:00.104222 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69955e7f-c79f-4f5a-b199-4584ad501dc3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:00.105391 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69955e7f-c79f-4f5a-b199-4584ad501dc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:00.106838 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69955e7f-c79f-4f5a-b199-4584ad501dc3 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:00.108622 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 512/1013] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:48:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:49:00.110137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:00.115073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69955e7f-c79f-4f5a-b199-4584ad501dc3 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:00.133260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c28c4-27c7-4cca-8a9e-22927e3b13b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:00.134186 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c28c4-27c7-4cca-8a9e-22927e3b13b0 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:00.134678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c28c4-27c7-4cca-8a9e-22927e3b13b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:00.135098 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107c28c4-27c7-4cca-8a9e-22927e3b13b0 None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:00.135964 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c28c4-27c7-4cca-8a9e-22927e3b13b0 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:00.136499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c28c4-27c7-4cca-8a9e-22927e3b13b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:00.138056 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107c28c4-27c7-4cca-8a9e-22927e3b13b0 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:00.148258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 502/1014] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 22:48:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:49:00.149278 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c28c4-27c7-4cca-8a9e-22927e3b13b0 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:00.195603 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:00.218829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69955e7f-c79f-4f5a-b199-4584ad501dc3 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:00.262979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c747aa5-d4ee-4821-9399-ba3e7608e394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:00.263725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c747aa5-d4ee-4821-9399-ba3e7608e394 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:00.264013 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c747aa5-d4ee-4821-9399-ba3e7608e394 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:00.264294 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c747aa5-d4ee-4821-9399-ba3e7608e394 None admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:00.264999 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c747aa5-d4ee-4821-9399-ba3e7608e394 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:00.266422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c747aa5-d4ee-4821-9399-ba3e7608e394 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:00.269815 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107c28c4-27c7-4cca-8a9e-22927e3b13b0 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:00.300454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 513/1015] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 22:49:00 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 22:49:00.300454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bacd5-d2dd-48d9-8f31-f7c6ecdbdf95 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:00.300454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bacd5-d2dd-48d9-8f31-f7c6ecdbdf95 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:00.300454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1bacd5-d2dd-48d9-8f31-f7c6ecdbdf95 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:00.300454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 514/1016] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:00.302716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33ad5c-563a-428e-a4a8-29f1d902b297 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:00.303071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33ad5c-563a-428e-a4a8-29f1d902b297 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:00.303338 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba33ad5c-563a-428e-a4a8-29f1d902b297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:00.309159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0839e7b6-79e6-433a-8d34-4820a4ed794f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:00.310213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839e7b6-79e6-433a-8d34-4820a4ed794f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:00.310506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839e7b6-79e6-433a-8d34-4820a4ed794f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:00.310837 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0839e7b6-79e6-433a-8d34-4820a4ed794f None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:00.311568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0839e7b6-79e6-433a-8d34-4820a4ed794f None admin] RBAC: Adding query filter params (type=compute) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:00.311884 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0839e7b6-79e6-433a-8d34-4820a4ed794f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:00.313295 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0839e7b6-79e6-433a-8d34-4820a4ed794f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:00.320944 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 503/1017] 10.222.0.17 () {56 vars in 1112 bytes} [Thu Jun 16 22:49:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:00.322359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-ba33ad5c-563a-428e-a4a8-29f1d902b297 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:00.323880 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ba33ad5c-563a-428e-a4a8-29f1d902b297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:00.327024 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0839e7b6-79e6-433a-8d34-4820a4ed794f None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:00.375446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:00.414640 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 515/1018] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:00 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:49:00.436504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33ad5c-563a-428e-a4a8-29f1d902b297 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:00.436822 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0839e7b6-79e6-433a-8d34-4820a4ed794f None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:00.476221 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1542c-b959-49cd-ba75-bef8ccdb782a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:00.477700 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1542c-b959-49cd-ba75-bef8ccdb782a None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:00.478148 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1542c-b959-49cd-ba75-bef8ccdb782a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:00.478693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1542c-b959-49cd-ba75-bef8ccdb782a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:00.479587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1542c-b959-49cd-ba75-bef8ccdb782a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:00.481111 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1542c-b959-49cd-ba75-bef8ccdb782a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:00.501523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 504/1019] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jun 16 22:49:00 2022] POST /identity/v3/endpoints => generated 334 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 22:49:00.511345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:00.559322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba33ad5c-563a-428e-a4a8-29f1d902b297 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:00.605427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27288ea6-5990-4076-8859-832246ae3906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:00.606143 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27288ea6-5990-4076-8859-832246ae3906 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:00.606393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27288ea6-5990-4076-8859-832246ae3906 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:00.606698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27288ea6-5990-4076-8859-832246ae3906 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:00.607425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27288ea6-5990-4076-8859-832246ae3906 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:00.608857 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27288ea6-5990-4076-8859-832246ae3906 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:00.625521 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 516/1020] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 22:49:00 2022] POST /identity/v3/services => generated 242 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 22:49:02.476221 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849a046-1083-42a5-9da6-903ceb679776 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:02.476753 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849a046-1083-42a5-9da6-903ceb679776 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:02.476850 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849a046-1083-42a5-9da6-903ceb679776 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:02.478199 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 505/1021] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:02.483314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de8de5-7968-4472-a841-a3ed63f62bf7 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:02.483687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de8de5-7968-4472-a841-a3ed63f62bf7 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:02.483847 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de8de5-7968-4472-a841-a3ed63f62bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:02.500561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-c5de8de5-7968-4472-a841-a3ed63f62bf7 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:02.501744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c5de8de5-7968-4472-a841-a3ed63f62bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:02.575677 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 517/1022] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:02.600427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849a046-1083-42a5-9da6-903ceb679776 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:02.704911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:02.711143 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849a046-1083-42a5-9da6-903ceb679776 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:02.739652 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d5475-d2b9-4357-9643-ff362c787111 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:02.739835 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d5475-d2b9-4357-9643-ff362c787111 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:02.740118 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d5475-d2b9-4357-9643-ff362c787111 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:02.742637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 518/1023] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:49:02.747384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6311c-0839-40f2-8c77-16aeca4494a3 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:02.747680 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6311c-0839-40f2-8c77-16aeca4494a3 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:02.747962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d6311c-0839-40f2-8c77-16aeca4494a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:02.752785 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1d26b-b60b-48bc-bf12-7e519f7d9a4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:02.753590 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1d26b-b60b-48bc-bf12-7e519f7d9a4c None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:02.753874 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1d26b-b60b-48bc-bf12-7e519f7d9a4c None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:02.754148 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba1d26b-b60b-48bc-bf12-7e519f7d9a4c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:02.755357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1d26b-b60b-48bc-bf12-7e519f7d9a4c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:02.756788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba1d26b-b60b-48bc-bf12-7e519f7d9a4c None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:02.758556 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 506/1024] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:49:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:02.764009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1d26b-b60b-48bc-bf12-7e519f7d9a4c None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:02.765577 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a8d6311c-0839-40f2-8c77-16aeca4494a3 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:02.766928 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a8d6311c-0839-40f2-8c77-16aeca4494a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:02.809821 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:02.834535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 519/1025] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:49:02.835930 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23191cf9-30c5-45cf-8d71-aa22dd1f40d5 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:02.836227 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23191cf9-30c5-45cf-8d71-aa22dd1f40d5 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:02.836508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23191cf9-30c5-45cf-8d71-aa22dd1f40d5 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:02.837776 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 520/1026] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:02.843026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023cddb-b1c6-4721-b7af-722eca71358c None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:02.843323 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023cddb-b1c6-4721-b7af-722eca71358c None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:02.843607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1023cddb-b1c6-4721-b7af-722eca71358c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:02.860526 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-1023cddb-b1c6-4721-b7af-722eca71358c None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:02.862222 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-1023cddb-b1c6-4721-b7af-722eca71358c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:02.866428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba1d26b-b60b-48bc-bf12-7e519f7d9a4c None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:02.898809 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3f982-d288-4cc3-bffb-cc3123ab2fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:02.899482 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3f982-d288-4cc3-bffb-cc3123ab2fda None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:02.899769 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3f982-d288-4cc3-bffb-cc3123ab2fda None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:02.900046 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb3f982-d288-4cc3-bffb-cc3123ab2fda None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:02.900742 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3f982-d288-4cc3-bffb-cc3123ab2fda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:02.902416 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb3f982-d288-4cc3-bffb-cc3123ab2fda None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:02.922558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 507/1027] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 22:49:02 2022] POST /identity/v3/users => generated 285 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 22:49:02.923380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3f982-d288-4cc3-bffb-cc3123ab2fda None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:02.929144 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 521/1028] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:02 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:49:02.952740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023cddb-b1c6-4721-b7af-722eca71358c None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:02.965852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:03.058356 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb3f982-d288-4cc3-bffb-cc3123ab2fda None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:03.071683 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1023cddb-b1c6-4721-b7af-722eca71358c None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:03.081128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:03.097566 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66b440-c07e-4aa3-8894-b4e965b1a0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:03.098399 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66b440-c07e-4aa3-8894-b4e965b1a0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:03.098716 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66b440-c07e-4aa3-8894-b4e965b1a0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:03.099016 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a66b440-c07e-4aa3-8894-b4e965b1a0c1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:03.112898 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a66b440-c07e-4aa3-8894-b4e965b1a0c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:49:03.115313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00bfff-aa4d-45c0-9374-fe55e0cebf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:03.116123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00bfff-aa4d-45c0-9374-fe55e0cebf0b None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:03.116413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00bfff-aa4d-45c0-9374-fe55e0cebf0b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:03.116694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00bfff-aa4d-45c0-9374-fe55e0cebf0b None admin] PATH_INFO: `/v3/services/image` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:03.117355 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00bfff-aa4d-45c0-9374-fe55e0cebf0b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:03.119061 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00bfff-aa4d-45c0-9374-fe55e0cebf0b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:03.121158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-6a66b440-c07e-4aa3-8894-b4e965b1a0c1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 22:49:03.121158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:03.121158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:03.121158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:03.121158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:03.121158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:03.121158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:03.121158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:03.121158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:03.121158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:03.121158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:03.121158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:03.121158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:49:03.121158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:49:03.121158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:49:03.121158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:49:03.122075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:49:03.122075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:49:03.122075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:49:03.122075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:49:03.122075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:03.122075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:03.122075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:03.122075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:03.122075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:03.122075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:03.122075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:03.122075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:03.122075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:03.122075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:03.122075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:03.122075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:03.123751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:03.123751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:03.123751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:03.123751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:03.123751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:49:03.123751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:49:03.123751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:49:03.123751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:49:03.123751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:49:03.123751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:49:03.123751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 22:49:03.123751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:49:03.123751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 508/1029] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jun 16 22:49:02 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:49:03.128649 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66b440-c07e-4aa3-8894-b4e965b1a0c1 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:03.132238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-ec00bfff-aa4d-45c0-9374-fe55e0cebf0b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 22:49:03.132238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:03.132238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:49:03.132238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:49:03.132238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:49:03.132238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:49:03.132238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:49:03.132238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:49:03.132238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 22:49:03.132238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:03.132238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:49:03.132238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:03.132238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:03.132238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:03.132238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:03.132238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:03.132238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:03.133401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:03.133401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:03.133401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:03.133401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:03.133401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:03.133401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:03.133401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:49:03.133401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:49:03.133401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:49:03.133401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:49:03.133401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:03.133401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:03.133401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:03.133401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:03.133401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:03.133401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:03.134500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:03.134500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:03.134500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:03.134500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:03.134500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:03.134500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:03.134500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:03.134500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:03.134500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:03.134500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:03.134500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:49:03.134500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:49:03.134500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 22:49:03.134500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:03.134500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 522/1030] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jun 16 22:49:02 2022] GET /identity/v3/services/image => generated 86 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:49:03.140291 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00bfff-aa4d-45c0-9374-fe55e0cebf0b None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:03.219519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:03.242171 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a66b440-c07e-4aa3-8894-b4e965b1a0c1 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:03.261710 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00bfff-aa4d-45c0-9374-fe55e0cebf0b None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:03.279235 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00addf0-2201-43ad-b3e7-d279cf4827e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:03.280140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00addf0-2201-43ad-b3e7-d279cf4827e4 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:03.280585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00addf0-2201-43ad-b3e7-d279cf4827e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:03.280868 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00addf0-2201-43ad-b3e7-d279cf4827e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:03.281613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00addf0-2201-43ad-b3e7-d279cf4827e4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:03.281926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00addf0-2201-43ad-b3e7-d279cf4827e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:03.283842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00addf0-2201-43ad-b3e7-d279cf4827e4 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:03.292214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 509/1031] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jun 16 22:49:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:49:03.298283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00addf0-2201-43ad-b3e7-d279cf4827e4 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:03.305370 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a36180-673c-44a6-92e0-11d512a05260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:03.306049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a36180-673c-44a6-92e0-11d512a05260 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:03.306338 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a36180-673c-44a6-92e0-11d512a05260 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:03.306649 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a36180-673c-44a6-92e0-11d512a05260 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:03.307391 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a36180-673c-44a6-92e0-11d512a05260 None admin] RBAC: Adding query filter params (name=image) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:03.308138 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a36180-673c-44a6-92e0-11d512a05260 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:03.309568 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a36180-673c-44a6-92e0-11d512a05260 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:03.316430 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 523/1032] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:49:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:49:03.321813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a36180-673c-44a6-92e0-11d512a05260 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:03.359350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:03.414003 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00addf0-2201-43ad-b3e7-d279cf4827e4 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:03.430648 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a36180-673c-44a6-92e0-11d512a05260 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:03.451002 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5bded-aeaa-45ce-a67f-a5899abcb0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:03.451914 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5bded-aeaa-45ce-a67f-a5899abcb0b8 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:03.452206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5bded-aeaa-45ce-a67f-a5899abcb0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:03.452484 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5bded-aeaa-45ce-a67f-a5899abcb0b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:03.465134 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5bded-aeaa-45ce-a67f-a5899abcb0b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:03.466600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5bded-aeaa-45ce-a67f-a5899abcb0b8 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:03.467477 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9847a-8e41-4da3-bbf2-2aff0548461a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:03.468149 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9847a-8e41-4da3-bbf2-2aff0548461a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:03.468441 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9847a-8e41-4da3-bbf2-2aff0548461a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:03.468720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f9847a-8e41-4da3-bbf2-2aff0548461a None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:03.469426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9847a-8e41-4da3-bbf2-2aff0548461a None admin] RBAC: Adding query filter params (type=image) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:03.469746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9847a-8e41-4da3-bbf2-2aff0548461a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:03.470280 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:03.471549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f9847a-8e41-4da3-bbf2-2aff0548461a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:03.476579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 524/1033] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:49:03 2022] GET /identity/v3/services?type=image => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:03.482249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9847a-8e41-4da3-bbf2-2aff0548461a None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:03.491973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-9af5bded-aeaa-45ce-a67f-a5899abcb0b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:49:03.491973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:03.491973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:03.491973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:03.491973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:03.491973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:03.491973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:03.491973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:03.491973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:03.491973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:03.491973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:03.491973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:03.491973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:49:03.491973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:49:03.491973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:49:03.493232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:49:03.493232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:03.493232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:03.493232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:03.493232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:03.493232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:03.493232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:03.493232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:03.493232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:03.493232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:03.493232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:03.493232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:03.493232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:03.493232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:03.493232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:03.493232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:03.493232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:03.494100 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:49:03.494100 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:49:03.494100 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:49:03.494100 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:49:03.494100 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 510/1034] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:49:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:49:03.498922 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5bded-aeaa-45ce-a67f-a5899abcb0b8 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:03.585769 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:03.590950 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f9847a-8e41-4da3-bbf2-2aff0548461a None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:03.606565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5bded-aeaa-45ce-a67f-a5899abcb0b8 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:03.627165 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7025e7-4ba6-4550-bd95-44ab1c18a39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:03.627825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7025e7-4ba6-4550-bd95-44ab1c18a39f None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:03.628160 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7025e7-4ba6-4550-bd95-44ab1c18a39f None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:03.628444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7025e7-4ba6-4550-bd95-44ab1c18a39f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:03.629237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7025e7-4ba6-4550-bd95-44ab1c18a39f None admin] RBAC: Adding query filter params (interface=public, service_id=8f9ef6e2d06a46b393fb4549d2abecd7, region_id=RegionOne) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:03.629554 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7025e7-4ba6-4550-bd95-44ab1c18a39f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:03.630949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7025e7-4ba6-4550-bd95-44ab1c18a39f None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:03.640817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 525/1035] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jun 16 22:49:03 2022] GET /identity/v3/endpoints?service_id=8f9ef6e2d06a46b393fb4549d2abecd7&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:49:03.641875 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7025e7-4ba6-4550-bd95-44ab1c18a39f None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:03.647651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d42e1-8ae5-47c6-b899-937964e8bc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:03.648329 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d42e1-8ae5-47c6-b899-937964e8bc8a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:03.648623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d42e1-8ae5-47c6-b899-937964e8bc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:03.648900 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173d42e1-8ae5-47c6-b899-937964e8bc8a None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:03.649619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d42e1-8ae5-47c6-b899-937964e8bc8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:03.649927 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d42e1-8ae5-47c6-b899-937964e8bc8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:03.652067 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-173d42e1-8ae5-47c6-b899-937964e8bc8a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:03.669592 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 511/1036] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:49:03.670480 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d42e1-8ae5-47c6-b899-937964e8bc8a None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:03.731028 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:03.746642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7025e7-4ba6-4550-bd95-44ab1c18a39f None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:03.779158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173d42e1-8ae5-47c6-b899-937964e8bc8a None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:03.781494 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0926a7-7cab-47cf-898f-97e5c1a35515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:03.782161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0926a7-7cab-47cf-898f-97e5c1a35515 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:03.782449 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0926a7-7cab-47cf-898f-97e5c1a35515 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:03.782789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0926a7-7cab-47cf-898f-97e5c1a35515 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:03.784071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0926a7-7cab-47cf-898f-97e5c1a35515 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:03.785429 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0926a7-7cab-47cf-898f-97e5c1a35515 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:03.792116 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 526/1037] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jun 16 22:49:03 2022] GET /identity/v3/services => generated 1957 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 22:49:03.817348 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313fc0c-4e60-4e5e-a129-2c6b234c6c72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:03.818179 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313fc0c-4e60-4e5e-a129-2c6b234c6c72 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:03.818466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313fc0c-4e60-4e5e-a129-2c6b234c6c72 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:03.818917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1313fc0c-4e60-4e5e-a129-2c6b234c6c72 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:03.827548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1313fc0c-4e60-4e5e-a129-2c6b234c6c72 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:03.829083 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1313fc0c-4e60-4e5e-a129-2c6b234c6c72 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:03.844981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-1313fc0c-4e60-4e5e-a129-2c6b234c6c72 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 22:49:03.844981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:03.844981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:03.844981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:03.844981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:03.844981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:03.844981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:03.844981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:03.844981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:03.844981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:03.844981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:03.844981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:03.844981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:49:03.844981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:49:03.844981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:49:03.844981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:49:03.847044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:03.847044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:03.847044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:49:03.847044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:49:03.847044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:49:03.847044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:49:03.847044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:03.847044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:03.847044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:03.847044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:03.847044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:03.847044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:03.847044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:03.847044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:03.847044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:03.847044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:03.847044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:03.847044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:03.847998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:03.847998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:03.847998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:49:03.847998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:49:03.847998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:49:03.847998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:49:03.847998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:49:03.847998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:49:03.847998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 22:49:03.847998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:49:03.847998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 512/1038] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jun 16 22:49:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:49:03.854823 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0926a7-7cab-47cf-898f-97e5c1a35515 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:03.886673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:03.969591 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0926a7-7cab-47cf-898f-97e5c1a35515 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:04.008427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26379a14-9340-4caf-aa52-a8d38b7f37b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:04.009383 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26379a14-9340-4caf-aa52-a8d38b7f37b7 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:04.009821 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26379a14-9340-4caf-aa52-a8d38b7f37b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:04.010245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26379a14-9340-4caf-aa52-a8d38b7f37b7 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:04.011270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26379a14-9340-4caf-aa52-a8d38b7f37b7 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:04.011731 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26379a14-9340-4caf-aa52-a8d38b7f37b7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:04.013315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26379a14-9340-4caf-aa52-a8d38b7f37b7 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:04.030922 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 527/1039] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jun 16 22:49:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:04.037279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313fc0c-4e60-4e5e-a129-2c6b234c6c72 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:04.142008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1313fc0c-4e60-4e5e-a129-2c6b234c6c72 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:04.159350 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:04.179016 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90898a9a-73ce-4056-8e16-bbecfe8999c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:04.179680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90898a9a-73ce-4056-8e16-bbecfe8999c6 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:04.179978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90898a9a-73ce-4056-8e16-bbecfe8999c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:04.180243 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90898a9a-73ce-4056-8e16-bbecfe8999c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:04.257233 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90898a9a-73ce-4056-8e16-bbecfe8999c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:04.258704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90898a9a-73ce-4056-8e16-bbecfe8999c6 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:04.362701 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 513/1040] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:49:04 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:49:04.364551 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26379a14-9340-4caf-aa52-a8d38b7f37b7 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:04.450849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:04.467252 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26379a14-9340-4caf-aa52-a8d38b7f37b7 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:04.501208 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc31b9-e783-4945-84ae-ebd5949804d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:04.502602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc31b9-e783-4945-84ae-ebd5949804d7 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:04.502905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc31b9-e783-4945-84ae-ebd5949804d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:04.503183 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdc31b9-e783-4945-84ae-ebd5949804d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:04.513253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc31b9-e783-4945-84ae-ebd5949804d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:04.514714 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdc31b9-e783-4945-84ae-ebd5949804d7 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:04.532401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-4fdc31b9-e783-4945-84ae-ebd5949804d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:49:04.532401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:04.532401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:04.532401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:04.532401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:04.532401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:04.532401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:04.532401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:04.532401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:04.532401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:04.532401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:04.532401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:04.532401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:49:04.532401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:49:04.532401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:49:04.533799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:49:04.533799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:04.533799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:04.533799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:04.533799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:04.533799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:04.533799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:04.533799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:04.533799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:04.533799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:04.533799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:04.533799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:04.533799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:04.533799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:04.533799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:04.533799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:04.533799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:04.535111 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:49:04.535111 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:49:04.535111 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:49:04.535111 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:04.538730 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 528/1041] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:49:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:49:04.540053 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90898a9a-73ce-4056-8e16-bbecfe8999c6 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:04.558005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:04.643877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90898a9a-73ce-4056-8e16-bbecfe8999c6 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:04.678872 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0033c0-c7b4-44a6-8ab6-77cd5158364b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:04.679542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0033c0-c7b4-44a6-8ab6-77cd5158364b None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:04.679827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0033c0-c7b4-44a6-8ab6-77cd5158364b None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:04.680105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0033c0-c7b4-44a6-8ab6-77cd5158364b None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:04.680818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0033c0-c7b4-44a6-8ab6-77cd5158364b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:04.681129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0033c0-c7b4-44a6-8ab6-77cd5158364b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:04.682484 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0033c0-c7b4-44a6-8ab6-77cd5158364b None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:04.694102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 514/1042] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:49:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:04.699724 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc31b9-e783-4945-84ae-ebd5949804d7 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:04.801277 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdc31b9-e783-4945-84ae-ebd5949804d7 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:04.834402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7bd3b-68f2-4d39-8f5e-1204d1c595d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:04.835233 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7bd3b-68f2-4d39-8f5e-1204d1c595d4 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:04.835519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7bd3b-68f2-4d39-8f5e-1204d1c595d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:04.835796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d7bd3b-68f2-4d39-8f5e-1204d1c595d4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:04.838530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:04.843894 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7bd3b-68f2-4d39-8f5e-1204d1c595d4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:04.845376 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d7bd3b-68f2-4d39-8f5e-1204d1c595d4 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:04.859805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-92d7bd3b-68f2-4d39-8f5e-1204d1c595d4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:49:04.859805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:04.859805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:04.859805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:04.859805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:04.859805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:04.859805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:04.859805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:04.859805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:04.859805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:04.859805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:04.859805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:04.859805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:49:04.859805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:49:04.859805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:49:04.860840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:49:04.860840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:04.860840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:04.860840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:04.860840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:04.860840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:04.860840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:04.860840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:04.860840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:04.860840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:04.860840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:04.860840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:04.860840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:04.860840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:04.860840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:04.860840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:04.860840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:04.862324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:49:04.862324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:49:04.862324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:49:04.862324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:49:04.862324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:49:04.862324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:49:04.862324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:49:04.862324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:04.866898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 529/1043] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:49:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:49:04.867987 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0033c0-c7b4-44a6-8ab6-77cd5158364b None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:04.949980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:04.974941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0033c0-c7b4-44a6-8ab6-77cd5158364b None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:05.009079 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a95a171-e9c8-486e-9347-7864680e56ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:05.009740 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95a171-e9c8-486e-9347-7864680e56ce None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:05.010025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95a171-e9c8-486e-9347-7864680e56ce None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:05.010302 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a95a171-e9c8-486e-9347-7864680e56ce None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:05.011121 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95a171-e9c8-486e-9347-7864680e56ce None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:05.011432 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95a171-e9c8-486e-9347-7864680e56ce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:05.012849 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a95a171-e9c8-486e-9347-7864680e56ce None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:05.026621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 515/1044] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:49:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:05.032282 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7bd3b-68f2-4d39-8f5e-1204d1c595d4 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:05.119475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5f2d9-b193-42e3-9551-2d4ecee7ccf7 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:05.119782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5f2d9-b193-42e3-9551-2d4ecee7ccf7 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:05.120033 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc5f2d9-b193-42e3-9551-2d4ecee7ccf7 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:05.121559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 516/1045] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:49:05.126026 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3c4cc-7c7b-4ead-a76e-61640038ec74 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:05.126314 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3c4cc-7c7b-4ead-a76e-61640038ec74 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:05.126613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b3c4cc-7c7b-4ead-a76e-61640038ec74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:05.138171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d7bd3b-68f2-4d39-8f5e-1204d1c595d4 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:05.145234 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-a5b3c4cc-7c7b-4ead-a76e-61640038ec74 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:05.146412 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-a5b3c4cc-7c7b-4ead-a76e-61640038ec74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:05.174992 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c6a97-15cf-4fa2-9577-f23e9c9957f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:05.175650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c6a97-15cf-4fa2-9577-f23e9c9957f0 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:05.175936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c6a97-15cf-4fa2-9577-f23e9c9957f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:05.176221 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867c6a97-15cf-4fa2-9577-f23e9c9957f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:05.177043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c6a97-15cf-4fa2-9577-f23e9c9957f0 None admin] RBAC: Adding query filter params (role.id=c6bedc2399d04c78a942f96283ed4560, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=4180f2593149437fa27effd8d1338d10) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:05.177353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c6a97-15cf-4fa2-9577-f23e9c9957f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:05.178784 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867c6a97-15cf-4fa2-9577-f23e9c9957f0 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:05.184132 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 530/1046] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:49:05 2022] GET /identity/v3/role_assignments?user.id=4180f2593149437fa27effd8d1338d10&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=c6bedc2399d04c78a942f96283ed4560 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:05.234640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 517/1047] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:49:05.255605 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c6a97-15cf-4fa2-9577-f23e9c9957f0 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:05.323717 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:05.372134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867c6a97-15cf-4fa2-9577-f23e9c9957f0 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:05.406393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366070f-cfd9-4c04-a336-92a0d160e91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:05.407226 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366070f-cfd9-4c04-a336-92a0d160e91c None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:05.407513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366070f-cfd9-4c04-a336-92a0d160e91c None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:05.407791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366070f-cfd9-4c04-a336-92a0d160e91c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:05.418523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366070f-cfd9-4c04-a336-92a0d160e91c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:49:05.426716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-5366070f-cfd9-4c04-a336-92a0d160e91c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:49:05.426716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:05.426716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:05.426716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:05.426716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:05.426716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:05.426716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:05.426716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:05.426716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:05.426716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:05.426716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:05.426716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:05.426716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:49:05.426716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:49:05.426716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:49:05.426716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:49:05.427636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:49:05.427636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:49:05.427636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:49:05.427636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:49:05.427636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:05.427636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:05.427636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:05.427636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:05.427636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:05.427636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:05.427636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:05.427636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:05.427636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:05.427636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:05.427636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:05.427636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:05.428504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:05.428504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:05.428504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:05.428504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:05.428504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:49:05.428504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:49:05.428504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:49:05.428504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:49:05.428504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:49:05.428504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:49:05.428504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:49:05.428504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:05.432978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 531/1048] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:49:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:49:05.434146 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3c4cc-7c7b-4ead-a76e-61640038ec74 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:05.500024 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:05.536364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b3c4cc-7c7b-4ead-a76e-61640038ec74 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:05.570428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadd27a-6bcd-4d8f-a3bb-20be4e1ea2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:05.571142 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadd27a-6bcd-4d8f-a3bb-20be4e1ea2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:05.571408 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadd27a-6bcd-4d8f-a3bb-20be4e1ea2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:05.571689 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadd27a-6bcd-4d8f-a3bb-20be4e1ea2c4 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:05.572429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadd27a-6bcd-4d8f-a3bb-20be4e1ea2c4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:05.572738 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadd27a-6bcd-4d8f-a3bb-20be4e1ea2c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:05.574101 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadd27a-6bcd-4d8f-a3bb-20be4e1ea2c4 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:05.587510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 518/1049] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:49:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:05.588746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366070f-cfd9-4c04-a336-92a0d160e91c None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:05.693052 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366070f-cfd9-4c04-a336-92a0d160e91c None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:05.715462 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:05.730468 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d800b00-f18c-428b-9b5c-0c33a608fe83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:05.731427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d800b00-f18c-428b-9b5c-0c33a608fe83 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:05.731714 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d800b00-f18c-428b-9b5c-0c33a608fe83 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:05.731990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d800b00-f18c-428b-9b5c-0c33a608fe83 None admin] PATH_INFO: `/v3/users/95cb7f9094484d55b47b6304995ab1fb` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:05.746249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d800b00-f18c-428b-9b5c-0c33a608fe83 None admin] RBAC: Authorizing `identity:get_user(user_id=95cb7f9094484d55b47b6304995ab1fb)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:05.747713 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d800b00-f18c-428b-9b5c-0c33a608fe83 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:05.754629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 532/1050] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:49:05 2022] GET /identity/v3/users/95cb7f9094484d55b47b6304995ab1fb => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:05.755743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadd27a-6bcd-4d8f-a3bb-20be4e1ea2c4 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:05.860213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadd27a-6bcd-4d8f-a3bb-20be4e1ea2c4 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:05.891291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:05.900079 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77255c4a-485b-4326-96f1-b681b68492ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:05.900737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77255c4a-485b-4326-96f1-b681b68492ad None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:05.901291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77255c4a-485b-4326-96f1-b681b68492ad None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:05.901573 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77255c4a-485b-4326-96f1-b681b68492ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:05.908281 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93795b9-1d3a-4d02-b4c4-c87541dcee92 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:05.908575 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93795b9-1d3a-4d02-b4c4-c87541dcee92 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:05.908926 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93795b9-1d3a-4d02-b4c4-c87541dcee92 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:05.910315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 533/1051] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:49:05.915102 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd6566-9c87-4ffb-98df-62897bf26099 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:05.915396 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd6566-9c87-4ffb-98df-62897bf26099 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:05.915679 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fd6566-9c87-4ffb-98df-62897bf26099 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:05.933567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-b5fd6566-9c87-4ffb-98df-62897bf26099 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:05.934893 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b5fd6566-9c87-4ffb-98df-62897bf26099 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:05.973126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77255c4a-485b-4326-96f1-b681b68492ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:05.974511 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77255c4a-485b-4326-96f1-b681b68492ad None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:06.007331 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 534/1052] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:49:06.030032 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd6566-9c87-4ffb-98df-62897bf26099 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:06.076243 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 519/1053] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:49:05 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:49:06.082359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77255c4a-485b-4326-96f1-b681b68492ad None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:06.109761 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:06.134440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fd6566-9c87-4ffb-98df-62897bf26099 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:06.144592 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:06.172293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0db27-3685-4570-ab6a-6ac4cd232c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:06.173402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0db27-3685-4570-ab6a-6ac4cd232c59 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:06.173402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0db27-3685-4570-ab6a-6ac4cd232c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:06.173732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0db27-3685-4570-ab6a-6ac4cd232c59 None admin] PATH_INFO: `/v3/services/image` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:06.174337 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0db27-3685-4570-ab6a-6ac4cd232c59 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:06.175723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0db27-3685-4570-ab6a-6ac4cd232c59 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:06.185615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77255c4a-485b-4326-96f1-b681b68492ad None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:06.187304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-70d0db27-3685-4570-ab6a-6ac4cd232c59 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 22:49:06.187304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:06.187304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:49:06.187304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:49:06.187304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:49:06.187304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:49:06.187304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:49:06.187304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:49:06.187304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 22:49:06.187304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:06.187304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:49:06.187304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:06.187304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:06.187304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:06.187304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:06.187304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:06.187304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:06.188255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:06.188255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:06.188255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:06.188255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:06.188255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:06.188255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:06.188255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:49:06.188255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:49:06.188255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:49:06.188255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:49:06.188255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:06.188255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:06.188255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:06.188255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:06.188255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:06.188255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:06.189643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:06.189643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:06.189643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:06.189643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:06.189643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:06.189643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:06.189643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:06.189643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:06.189643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:06.189643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:06.189643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:49:06.189643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:49:06.189643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 22:49:06.189643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:06.189643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 535/1054] 10.222.0.17 () {56 vars in 1105 bytes} [Thu Jun 16 22:49:06 2022] GET /identity/v3/services/image => generated 86 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:49:06.194735 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0db27-3685-4570-ab6a-6ac4cd232c59 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:06.227590 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b24390-13d2-42b6-b2ce-4f793cd6a53d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:06.228378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b24390-13d2-42b6-b2ce-4f793cd6a53d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:06.228668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b24390-13d2-42b6-b2ce-4f793cd6a53d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:06.228946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b24390-13d2-42b6-b2ce-4f793cd6a53d None admin] PATH_INFO: `/v3/projects/6b1cad07ad694f78ad30147bda444ce5` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:06.230132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b24390-13d2-42b6-b2ce-4f793cd6a53d None admin] RBAC: Authorizing `identity:get_project(project_id=6b1cad07ad694f78ad30147bda444ce5)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:06.231600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b24390-13d2-42b6-b2ce-4f793cd6a53d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:06.233615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 520/1055] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:49:06 2022] GET /identity/v3/projects/6b1cad07ad694f78ad30147bda444ce5 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:49:06.239359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b24390-13d2-42b6-b2ce-4f793cd6a53d None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:06.306570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0db27-3685-4570-ab6a-6ac4cd232c59 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:06.343171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94ef22-f3a3-42d7-8108-bcdf92a6a3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:06.343961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94ef22-f3a3-42d7-8108-bcdf92a6a3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:06.344269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94ef22-f3a3-42d7-8108-bcdf92a6a3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:06.344597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:06.345189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de94ef22-f3a3-42d7-8108-bcdf92a6a3d6 None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:06.346444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94ef22-f3a3-42d7-8108-bcdf92a6a3d6 None admin] RBAC: Adding query filter params (name=image) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:06.346783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94ef22-f3a3-42d7-8108-bcdf92a6a3d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:06.348905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de94ef22-f3a3-42d7-8108-bcdf92a6a3d6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:06.352358 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b24390-13d2-42b6-b2ce-4f793cd6a53d None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:06.358817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 536/1056] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:49:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:49:06.359804 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94ef22-f3a3-42d7-8108-bcdf92a6a3d6 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:06.392063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064ae9a-43bd-4b67-9358-99db23e45c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:06.393306 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064ae9a-43bd-4b67-9358-99db23e45c13 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:06.393593 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064ae9a-43bd-4b67-9358-99db23e45c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:06.393870 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064ae9a-43bd-4b67-9358-99db23e45c13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:06.394731 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064ae9a-43bd-4b67-9358-99db23e45c13 None admin] RBAC: Adding query filter params (role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.project.id=6b1cad07ad694f78ad30147bda444ce5, user.id=95cb7f9094484d55b47b6304995ab1fb) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:06.395047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064ae9a-43bd-4b67-9358-99db23e45c13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:06.396484 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064ae9a-43bd-4b67-9358-99db23e45c13 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:06.397865 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:06.410623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 521/1057] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:49:06 2022] GET /identity/v3/role_assignments?user.id=95cb7f9094484d55b47b6304995ab1fb&scope.project.id=6b1cad07ad694f78ad30147bda444ce5&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:49:06.480242 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de94ef22-f3a3-42d7-8108-bcdf92a6a3d6 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:06.516061 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3f1e41-1ad5-4b99-b795-36615b24385e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:06.516746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f1e41-1ad5-4b99-b795-36615b24385e None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:06.517033 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f1e41-1ad5-4b99-b795-36615b24385e None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:06.517854 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3f1e41-1ad5-4b99-b795-36615b24385e None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:06.518579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f1e41-1ad5-4b99-b795-36615b24385e None admin] RBAC: Adding query filter params (type=image) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:06.518926 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f1e41-1ad5-4b99-b795-36615b24385e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:06.520329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3f1e41-1ad5-4b99-b795-36615b24385e None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:06.526176 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 537/1058] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:49:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:06.532311 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064ae9a-43bd-4b67-9358-99db23e45c13 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:06.635212 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064ae9a-43bd-4b67-9358-99db23e45c13 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:06.669458 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ae8ede-fea4-473e-8bfc-5872584a0e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:06.670430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae8ede-fea4-473e-8bfc-5872584a0e1c None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:06.670949 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae8ede-fea4-473e-8bfc-5872584a0e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:06.671412 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ae8ede-fea4-473e-8bfc-5872584a0e1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:06.672331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae8ede-fea4-473e-8bfc-5872584a0e1c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:06.674423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ae8ede-fea4-473e-8bfc-5872584a0e1c None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:06.694409 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 522/1059] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jun 16 22:49:06 2022] POST /identity/v3/endpoints => generated 327 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:49:07.306401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b5ca7-1464-4eda-9eed-b5d05578d64d None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:07.306962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b5ca7-1464-4eda-9eed-b5d05578d64d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:07.307041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b5ca7-1464-4eda-9eed-b5d05578d64d None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:07.308402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 538/1060] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:07.313606 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2202c04-60e5-4031-9ae5-4994331b090c None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:07.313902 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2202c04-60e5-4031-9ae5-4994331b090c None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:07.314290 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2202c04-60e5-4031-9ae5-4994331b090c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:07.331345 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-f2202c04-60e5-4031-9ae5-4994331b090c None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:07.332518 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f2202c04-60e5-4031-9ae5-4994331b090c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:07.343654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:07.410296 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 523/1061] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:07.433121 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b5ca7-1464-4eda-9eed-b5d05578d64d None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:07.537465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:07.542521 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434b5ca7-1464-4eda-9eed-b5d05578d64d None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:07.580450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26919c96-a3e8-43a6-9ade-38eba3ae5e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:07.581269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26919c96-a3e8-43a6-9ade-38eba3ae5e7d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:07.581554 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26919c96-a3e8-43a6-9ade-38eba3ae5e7d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:07.581830 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26919c96-a3e8-43a6-9ade-38eba3ae5e7d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:07.593129 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26919c96-a3e8-43a6-9ade-38eba3ae5e7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:49:07.601732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-26919c96-a3e8-43a6-9ade-38eba3ae5e7d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 22:49:07.601732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:07.601732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:07.601732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:07.601732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:07.601732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:07.601732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:07.601732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:07.601732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:07.601732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:07.601732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:07.601732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:07.601732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:49:07.601732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:49:07.601732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:49:07.601732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:49:07.602729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:49:07.602729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:49:07.602729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:49:07.602729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:49:07.602729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:07.602729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:07.602729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:07.602729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:07.602729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:07.602729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:07.602729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:07.602729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:07.602729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:07.602729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:07.602729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:07.602729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:07.603559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:07.603559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:07.603559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:07.603559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:07.603559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:49:07.603559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:49:07.603559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:49:07.603559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:49:07.603559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:49:07.603559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:49:07.603559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 22:49:07.603559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:07.603559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 539/1062] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jun 16 22:49:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:49:07.610412 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2202c04-60e5-4031-9ae5-4994331b090c None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:07.711316 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2202c04-60e5-4031-9ae5-4994331b090c None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:07.744207 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40046e5-881d-446c-bca0-e344b962b720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:07.744875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40046e5-881d-446c-bca0-e344b962b720 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:07.745158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40046e5-881d-446c-bca0-e344b962b720 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:07.745434 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40046e5-881d-446c-bca0-e344b962b720 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:07.746167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40046e5-881d-446c-bca0-e344b962b720 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:07.746476 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40046e5-881d-446c-bca0-e344b962b720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:07.747864 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40046e5-881d-446c-bca0-e344b962b720 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:07.756520 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 524/1063] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jun 16 22:49:07 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:49:07.761982 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26919c96-a3e8-43a6-9ade-38eba3ae5e7d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:07.863576 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26919c96-a3e8-43a6-9ade-38eba3ae5e7d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:07.897291 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c2562-aa4d-45b2-9a15-c732ea93f33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:07.898091 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c2562-aa4d-45b2-9a15-c732ea93f33a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:07.898377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c2562-aa4d-45b2-9a15-c732ea93f33a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:07.898662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c2562-aa4d-45b2-9a15-c732ea93f33a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:07.910042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c2562-aa4d-45b2-9a15-c732ea93f33a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:07.911502 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900c2562-aa4d-45b2-9a15-c732ea93f33a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:07.928958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-900c2562-aa4d-45b2-9a15-c732ea93f33a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:49:07.928958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:07.928958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:07.928958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:07.928958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:07.928958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:07.928958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:07.928958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:07.928958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:07.928958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:07.928958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:07.928958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:07.928958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:49:07.928958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:49:07.928958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:49:07.929872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:49:07.929872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:07.929872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:07.929872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:07.929872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:07.929872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:07.929872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:07.929872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:07.929872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:07.929872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:07.929872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:07.929872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:07.929872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:07.929872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:07.929872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:07.929872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:07.929872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:07.931203 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:49:07.931203 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:49:07.931203 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:49:07.931203 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:07.934749 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 540/1064] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:49:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:49:07.939259 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40046e5-881d-446c-bca0-e344b962b720 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:07.943156 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:08.046911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40046e5-881d-446c-bca0-e344b962b720 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:08.081407 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673c17f-8503-41c4-84b3-5818447308eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:08.082081 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673c17f-8503-41c4-84b3-5818447308eb None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:08.082369 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673c17f-8503-41c4-84b3-5818447308eb None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:08.082678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9673c17f-8503-41c4-84b3-5818447308eb None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:08.083395 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673c17f-8503-41c4-84b3-5818447308eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:08.083706 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673c17f-8503-41c4-84b3-5818447308eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:08.085071 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9673c17f-8503-41c4-84b3-5818447308eb None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:08.101569 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 525/1065] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:49:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:08.103509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c2562-aa4d-45b2-9a15-c732ea93f33a None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:08.119767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:08.212865 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900c2562-aa4d-45b2-9a15-c732ea93f33a None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:08.251967 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4b67f-9b00-4e27-a31f-74b214ddb72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:08.252777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4b67f-9b00-4e27-a31f-74b214ddb72d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:08.253065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4b67f-9b00-4e27-a31f-74b214ddb72d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:08.253344 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def4b67f-9b00-4e27-a31f-74b214ddb72d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:08.261138 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4b67f-9b00-4e27-a31f-74b214ddb72d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:08.262581 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def4b67f-9b00-4e27-a31f-74b214ddb72d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:08.276954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-def4b67f-9b00-4e27-a31f-74b214ddb72d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 22:49:08.276954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:08.276954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:08.276954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:08.276954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:08.276954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:08.276954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:08.276954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:08.276954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:08.276954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:08.276954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:08.276954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:08.276954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:49:08.276954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:49:08.276954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:49:08.276954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:49:08.277881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:08.277881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:08.277881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:49:08.277881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:49:08.277881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:49:08.277881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:49:08.277881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:08.277881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:08.277881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:08.277881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:08.277881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:08.277881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:08.277881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:08.277881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:08.277881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:08.277881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:08.277881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:08.277881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:08.278940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:08.278940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:08.278940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:49:08.278940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:49:08.278940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:49:08.278940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:49:08.278940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:49:08.278940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:49:08.278940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 22:49:08.278940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:08.283437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 541/1066] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jun 16 22:49:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:49:08.284666 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673c17f-8503-41c4-84b3-5818447308eb None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:08.347601 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:08.387159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9673c17f-8503-41c4-84b3-5818447308eb None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:08.422314 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb8889b-34a3-4b72-adac-8f08b42e6062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:08.423034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8889b-34a3-4b72-adac-8f08b42e6062 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:08.423385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8889b-34a3-4b72-adac-8f08b42e6062 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:08.423668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb8889b-34a3-4b72-adac-8f08b42e6062 None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:08.425010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb8889b-34a3-4b72-adac-8f08b42e6062 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:08.425323 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb8889b-34a3-4b72-adac-8f08b42e6062 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:08.426775 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb8889b-34a3-4b72-adac-8f08b42e6062 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:08.448330 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 526/1067] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jun 16 22:49:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:08.450007 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4b67f-9b00-4e27-a31f-74b214ddb72d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:08.527215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:08.543722 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6e05d-ce4c-4882-854f-9bc044d64271 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:08.544014 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6e05d-ce4c-4882-854f-9bc044d64271 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:08.544294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6e05d-ce4c-4882-854f-9bc044d64271 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:08.545803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 527/1068] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:49:08.550954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537d095-26f1-451e-be2d-db97adf41814 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:08.551248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537d095-26f1-451e-be2d-db97adf41814 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:08.551532 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3537d095-26f1-451e-be2d-db97adf41814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:08.561109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def4b67f-9b00-4e27-a31f-74b214ddb72d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:08.567879 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-3537d095-26f1-451e-be2d-db97adf41814 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:08.569236 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-3537d095-26f1-451e-be2d-db97adf41814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:08.597429 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66416ee1-c33d-412a-b355-3287de69abde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:08.598229 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66416ee1-c33d-412a-b355-3287de69abde None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:08.598520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66416ee1-c33d-412a-b355-3287de69abde None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:08.598839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66416ee1-c33d-412a-b355-3287de69abde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:08.614147 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66416ee1-c33d-412a-b355-3287de69abde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:08.615644 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66416ee1-c33d-412a-b355-3287de69abde None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:08.632849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-66416ee1-c33d-412a-b355-3287de69abde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:49:08.632849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:08.632849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:08.632849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:08.632849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:08.632849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:08.632849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:08.632849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:08.632849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:08.632849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:08.632849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:08.632849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:08.632849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:49:08.632849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:49:08.632849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:49:08.633773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:49:08.633773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:08.633773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:08.633773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:08.633773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:08.633773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:08.633773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:08.633773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:08.633773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:08.633773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:08.633773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:08.633773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:08.633773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:08.633773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:08.633773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:08.633773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:08.633773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:08.635369 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:49:08.635369 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:49:08.635369 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:49:08.635369 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:08.635369 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 542/1069] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:49:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:49:08.639748 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66416ee1-c33d-412a-b355-3287de69abde None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:08.639870 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 528/1070] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:49:08.662695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3537d095-26f1-451e-be2d-db97adf41814 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:08.715518 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:08.745614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66416ee1-c33d-412a-b355-3287de69abde None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:08.765764 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3537d095-26f1-451e-be2d-db97adf41814 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:08.769545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:08.784652 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022c5b1-57f0-4fd5-8d4b-2754dbf20a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:08.785329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022c5b1-57f0-4fd5-8d4b-2754dbf20a81 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:08.785622 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022c5b1-57f0-4fd5-8d4b-2754dbf20a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:08.786116 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a022c5b1-57f0-4fd5-8d4b-2754dbf20a81 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:08.786663 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a022c5b1-57f0-4fd5-8d4b-2754dbf20a81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:08.786982 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a022c5b1-57f0-4fd5-8d4b-2754dbf20a81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:08.788380 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a022c5b1-57f0-4fd5-8d4b-2754dbf20a81 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:08.807385 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 543/1071] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:49:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:49:08.808594 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022c5b1-57f0-4fd5-8d4b-2754dbf20a81 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:08.810548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea5d77-a0b7-46bd-a5e6-e949898c09c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:08.811425 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea5d77-a0b7-46bd-a5e6-e949898c09c8 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:08.811721 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea5d77-a0b7-46bd-a5e6-e949898c09c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:08.812007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ea5d77-a0b7-46bd-a5e6-e949898c09c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:08.824878 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ea5d77-a0b7-46bd-a5e6-e949898c09c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:49:08.833139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-f1ea5d77-a0b7-46bd-a5e6-e949898c09c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:49:08.833139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:08.833139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:08.833139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:08.833139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:08.833139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:08.833139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:08.833139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:08.833139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:08.833139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:08.833139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:08.833139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:08.833139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:49:08.833139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:49:08.833139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:49:08.833139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:49:08.834106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:49:08.834106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:49:08.834106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:49:08.834106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:49:08.834106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:08.834106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:08.834106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:08.834106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:08.834106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:08.834106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:08.834106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:08.834106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:08.834106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:08.834106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:08.834106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:08.834106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:08.835010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:08.835010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:08.835010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:08.835010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:08.835010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:49:08.835010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:49:08.835010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:49:08.835010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:49:08.835010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:49:08.835010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:49:08.835010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:49:08.835010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:49:08.839654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 529/1072] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:49:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:49:08.840306 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea5d77-a0b7-46bd-a5e6-e949898c09c8 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:08.914984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a022c5b1-57f0-4fd5-8d4b-2754dbf20a81 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:08.941184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ea5d77-a0b7-46bd-a5e6-e949898c09c8 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:08.947369 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896298f1-8954-4634-8c0b-9703c137a764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:08.948162 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896298f1-8954-4634-8c0b-9703c137a764 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:08.948447 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896298f1-8954-4634-8c0b-9703c137a764 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:08.948725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896298f1-8954-4634-8c0b-9703c137a764 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:08.953778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:08.955954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896298f1-8954-4634-8c0b-9703c137a764 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:08.957377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896298f1-8954-4634-8c0b-9703c137a764 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:08.971510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-896298f1-8954-4634-8c0b-9703c137a764 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:49:08.971510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:08.971510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:08.971510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:08.971510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:08.971510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:08.971510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:08.971510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:08.971510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:08.971510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:08.971510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:08.971510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:08.971510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:49:08.971510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:49:08.971510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:49:08.972480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:49:08.972480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:08.972480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:08.972480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:08.972480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:08.972480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:08.972480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:08.972480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:08.972480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:08.972480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:08.972480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:08.972480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:08.972480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:08.972480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:08.972480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:08.972480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:08.972480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:08.973390 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:49:08.973390 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:49:08.973390 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:49:08.973390 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:49:08.973390 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:49:08.973390 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:49:08.973390 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:49:08.973390 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:08.973390 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 544/1073] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:49:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:49:08.976097 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd2528-612d-44bf-b1c4-eafe43ee1077 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:08.976669 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd2528-612d-44bf-b1c4-eafe43ee1077 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:08.976961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd2528-612d-44bf-b1c4-eafe43ee1077 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:08.977224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645314ea-7af7-4665-adc5-80119aa2808e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:08.977965 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645314ea-7af7-4665-adc5-80119aa2808e None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:08.978256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645314ea-7af7-4665-adc5-80119aa2808e None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:08.978531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 545/1074] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:08.978673 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645314ea-7af7-4665-adc5-80119aa2808e None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:08.979261 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd2528-612d-44bf-b1c4-eafe43ee1077 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:08.979364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645314ea-7af7-4665-adc5-80119aa2808e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:08.979657 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645314ea-7af7-4665-adc5-80119aa2808e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:08.981189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645314ea-7af7-4665-adc5-80119aa2808e None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:08.995112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 530/1075] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:49:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:49:08.995112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78db15f-3af6-4fd3-b435-719a544d9640 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:08.995112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78db15f-3af6-4fd3-b435-719a544d9640 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:08.995112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78db15f-3af6-4fd3-b435-719a544d9640 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:09.009711 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-f78db15f-3af6-4fd3-b435-719a544d9640 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:09.010963 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f78db15f-3af6-4fd3-b435-719a544d9640 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:09.035734 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:09.076809 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 531/1076] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:09.077836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78db15f-3af6-4fd3-b435-719a544d9640 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:09.091134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd2528-612d-44bf-b1c4-eafe43ee1077 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:09.126536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb234b-2c9c-4f6a-b284-3d28e91380fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:09.127237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb234b-2c9c-4f6a-b284-3d28e91380fb None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:09.127533 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb234b-2c9c-4f6a-b284-3d28e91380fb None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:09.127822 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ceb234b-2c9c-4f6a-b284-3d28e91380fb None admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:09.128632 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb234b-2c9c-4f6a-b284-3d28e91380fb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:09.128951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb234b-2c9c-4f6a-b284-3d28e91380fb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:09.130374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ceb234b-2c9c-4f6a-b284-3d28e91380fb None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:09.143083 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 546/1077] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:49:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:09.143887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb234b-2c9c-4f6a-b284-3d28e91380fb None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:09.175242 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78db15f-3af6-4fd3-b435-719a544d9640 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:09.178358 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:09.209983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa8994-6510-4f8a-b98d-802e088e07dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:09.210971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa8994-6510-4f8a-b98d-802e088e07dd None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:09.211259 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa8994-6510-4f8a-b98d-802e088e07dd None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:09.211538 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43aa8994-6510-4f8a-b98d-802e088e07dd None admin] PATH_INFO: `/v3/users/95cb7f9094484d55b47b6304995ab1fb` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:09.212743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa8994-6510-4f8a-b98d-802e088e07dd None admin] RBAC: Authorizing `identity:get_user(user_id=95cb7f9094484d55b47b6304995ab1fb)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:09.214169 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43aa8994-6510-4f8a-b98d-802e088e07dd None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:09.215853 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 532/1078] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:49:09 2022] GET /identity/v3/users/95cb7f9094484d55b47b6304995ab1fb => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:09.216582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa8994-6510-4f8a-b98d-802e088e07dd None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:09.244607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ceb234b-2c9c-4f6a-b284-3d28e91380fb None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:09.247239 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:09.280685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe7dfe-d85d-4f4e-bab4-b0858f1806d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:09.281324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe7dfe-d85d-4f4e-bab4-b0858f1806d4 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:09.281609 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe7dfe-d85d-4f4e-bab4-b0858f1806d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:09.281886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebe7dfe-d85d-4f4e-bab4-b0858f1806d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:09.316215 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43aa8994-6510-4f8a-b98d-802e088e07dd None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:09.347964 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe7dfe-d85d-4f4e-bab4-b0858f1806d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:09.350020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b08194-5752-467d-821d-e11694496462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:09.350209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebe7dfe-d85d-4f4e-bab4-b0858f1806d4 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:09.350943 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b08194-5752-467d-821d-e11694496462 None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:09.351230 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b08194-5752-467d-821d-e11694496462 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:09.351507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b08194-5752-467d-821d-e11694496462 None admin] PATH_INFO: `/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/4180f2593149437fa27effd8d1338d10/roles/c6bedc2399d04c78a942f96283ed4560` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:09.353746 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b08194-5752-467d-821d-e11694496462 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc48914270df4db996bd2a4b5b9fc777, user_id=4180f2593149437fa27effd8d1338d10, role_id=c6bedc2399d04c78a942f96283ed4560)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:09.355214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b08194-5752-467d-821d-e11694496462 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:09.371568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 533/1079] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:49:09 2022] PUT /identity/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/4180f2593149437fa27effd8d1338d10/roles/c6bedc2399d04c78a942f96283ed4560 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:49:09.372378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b08194-5752-467d-821d-e11694496462 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:09.422521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:09.469655 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 547/1080] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:49:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:49:09.475507 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe7dfe-d85d-4f4e-bab4-b0858f1806d4 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:09.479573 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:09.504134 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b08194-5752-467d-821d-e11694496462 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:09.548665 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f436b9-bf0e-4f68-942b-c0276e000f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:09.549539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f436b9-bf0e-4f68-942b-c0276e000f6d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:09.549822 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f436b9-bf0e-4f68-942b-c0276e000f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:09.550099 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f436b9-bf0e-4f68-942b-c0276e000f6d None admin] PATH_INFO: `/v3/projects/6b1cad07ad694f78ad30147bda444ce5` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:09.551373 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f436b9-bf0e-4f68-942b-c0276e000f6d None admin] RBAC: Authorizing `identity:get_project(project_id=6b1cad07ad694f78ad30147bda444ce5)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:09.552834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f436b9-bf0e-4f68-942b-c0276e000f6d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:09.554889 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 534/1081] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:49:09 2022] GET /identity/v3/projects/6b1cad07ad694f78ad30147bda444ce5 => generated 311 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:09.561206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f436b9-bf0e-4f68-942b-c0276e000f6d None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:09.592842 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebe7dfe-d85d-4f4e-bab4-b0858f1806d4 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:09.629273 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e199bf1-3187-482e-99c4-a758e5c3a25a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:09.630083 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e199bf1-3187-482e-99c4-a758e5c3a25a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:09.630387 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e199bf1-3187-482e-99c4-a758e5c3a25a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:09.630689 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e199bf1-3187-482e-99c4-a758e5c3a25a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:09.641389 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e199bf1-3187-482e-99c4-a758e5c3a25a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:09.642892 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e199bf1-3187-482e-99c4-a758e5c3a25a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:09.662083 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:09.663104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-4e199bf1-3187-482e-99c4-a758e5c3a25a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:49:09.663104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:09.663104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:09.663104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:09.663104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:09.663104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:09.663104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:09.663104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:09.663104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:09.663104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:09.663104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:09.663104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:09.663104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:49:09.663104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:49:09.663104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:49:09.664529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:49:09.664529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:09.664529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:09.664529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:09.664529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:09.664529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:09.664529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:09.664529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:09.664529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:09.664529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:09.664529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:09.664529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:09.664529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:09.664529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:09.664529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:09.664529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:09.664529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:09.665464 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:49:09.665464 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:49:09.665464 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:49:09.665464 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:09.667768 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f436b9-bf0e-4f68-942b-c0276e000f6d None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:09.670050 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 548/1082] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:49:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:49:09.671298 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e199bf1-3187-482e-99c4-a758e5c3a25a None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:09.707515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f1b604-76b8-4b23-9cc5-5bbf73df2e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:09.708406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1b604-76b8-4b23-9cc5-5bbf73df2e91 None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:09.708700 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1b604-76b8-4b23-9cc5-5bbf73df2e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:09.708975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f1b604-76b8-4b23-9cc5-5bbf73df2e91 None admin] PATH_INFO: `/v3/projects/6b1cad07ad694f78ad30147bda444ce5/users/95cb7f9094484d55b47b6304995ab1fb/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:09.711033 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1b604-76b8-4b23-9cc5-5bbf73df2e91 None admin] RBAC: Authorizing `identity:create_grant(project_id=6b1cad07ad694f78ad30147bda444ce5, user_id=95cb7f9094484d55b47b6304995ab1fb, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:09.712460 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f1b604-76b8-4b23-9cc5-5bbf73df2e91 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:09.726314 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 535/1083] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:49:09 2022] PUT /identity/v3/projects/6b1cad07ad694f78ad30147bda444ce5/users/95cb7f9094484d55b47b6304995ab1fb/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 22:49:09.738252 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:09.791350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e199bf1-3187-482e-99c4-a758e5c3a25a None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:09.829154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0ca065-0a8a-43a1-8244-8a1cca7af984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:09.829944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ca065-0a8a-43a1-8244-8a1cca7af984 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:09.830234 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ca065-0a8a-43a1-8244-8a1cca7af984 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:09.830509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0ca065-0a8a-43a1-8244-8a1cca7af984 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:09.831272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ca065-0a8a-43a1-8244-8a1cca7af984 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:09.831581 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ca065-0a8a-43a1-8244-8a1cca7af984 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:09.832963 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0ca065-0a8a-43a1-8244-8a1cca7af984 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:09.850949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 549/1084] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:49:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:49:11.519694 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11725b-eb18-4ece-904c-8b8ac743a983 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:11.520188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11725b-eb18-4ece-904c-8b8ac743a983 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:11.520188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c11725b-eb18-4ece-904c-8b8ac743a983 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:11.521533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 536/1085] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:11.526993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6422c3-ca3c-4916-8eac-fae505f625df None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:11.527405 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6422c3-ca3c-4916-8eac-fae505f625df None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:11.527549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6422c3-ca3c-4916-8eac-fae505f625df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:11.544308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-1b6422c3-ca3c-4916-8eac-fae505f625df None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:11.545475 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-1b6422c3-ca3c-4916-8eac-fae505f625df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:11.611055 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 550/1086] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:11.635261 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11725b-eb18-4ece-904c-8b8ac743a983 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:11.736945 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c11725b-eb18-4ece-904c-8b8ac743a983 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:11.769533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee3efbe-af24-42a8-a76b-3e027e930183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:11.770387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3efbe-af24-42a8-a76b-3e027e930183 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:11.770649 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3efbe-af24-42a8-a76b-3e027e930183 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:11.770931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee3efbe-af24-42a8-a76b-3e027e930183 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:11.781043 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee3efbe-af24-42a8-a76b-3e027e930183 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:49:11.789427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-6ee3efbe-af24-42a8-a76b-3e027e930183 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 22:49:11.789427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:11.789427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:11.789427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:11.789427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:11.789427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:11.789427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:11.789427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:11.789427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:11.789427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:11.789427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:11.789427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:11.789427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:49:11.789427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:49:11.789427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:49:11.789427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:49:11.790646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:49:11.790646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:49:11.790646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:49:11.790646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:49:11.790646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:11.790646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:11.790646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:11.790646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:11.790646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:11.790646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:11.790646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:11.790646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:11.790646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:11.790646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:11.790646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:11.790646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:11.791691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:11.791691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:11.791691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:11.791691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:11.791691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:49:11.791691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:49:11.791691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:49:11.791691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:49:11.791691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:49:11.791691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:49:11.791691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 22:49:11.791691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:49:11.794800 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 537/1087] 10.222.0.17 () {56 vars in 1120 bytes} [Thu Jun 16 22:49:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:49:11.796869 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6422c3-ca3c-4916-8eac-fae505f625df None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:11.845371 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:11.854233 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6477623-730a-470d-9e37-00395daaf34a None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:11.854530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6477623-730a-470d-9e37-00395daaf34a None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:11.854860 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6477623-730a-470d-9e37-00395daaf34a None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:11.856235 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 538/1088] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:49:11.861201 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc8e11-6a74-4be7-a00a-a2087ea2104d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:11.861495 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc8e11-6a74-4be7-a00a-a2087ea2104d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:11.861784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc8e11-6a74-4be7-a00a-a2087ea2104d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:11.878103 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-10cc8e11-6a74-4be7-a00a-a2087ea2104d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:11.879328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-10cc8e11-6a74-4be7-a00a-a2087ea2104d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:11.893460 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:11.898464 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6422c3-ca3c-4916-8eac-fae505f625df None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:11.932699 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df7056-f933-45f5-9729-0bd667542e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:11.933363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df7056-f933-45f5-9729-0bd667542e2c None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:11.933650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df7056-f933-45f5-9729-0bd667542e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:11.933922 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16df7056-f933-45f5-9729-0bd667542e2c None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:11.934682 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df7056-f933-45f5-9729-0bd667542e2c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:11.934990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df7056-f933-45f5-9729-0bd667542e2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:11.936339 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16df7056-f933-45f5-9729-0bd667542e2c None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:11.944281 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 551/1089] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jun 16 22:49:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:11.949320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 539/1090] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:49:11.950755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df7056-f933-45f5-9729-0bd667542e2c None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:11.973703 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc8e11-6a74-4be7-a00a-a2087ea2104d None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:12.050371 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16df7056-f933-45f5-9729-0bd667542e2c None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:12.059027 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:12.070429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cc8e11-6a74-4be7-a00a-a2087ea2104d None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:12.084762 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eaeeb8-422c-4e69-b3a3-668aa36da385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:12.085549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eaeeb8-422c-4e69-b3a3-668aa36da385 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:12.085835 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eaeeb8-422c-4e69-b3a3-668aa36da385 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:12.086109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36eaeeb8-422c-4e69-b3a3-668aa36da385 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:12.096407 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eaeeb8-422c-4e69-b3a3-668aa36da385 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:12.097826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36eaeeb8-422c-4e69-b3a3-668aa36da385 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:12.103429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ee114-6c61-4775-b8d1-e481700bcf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:12.104212 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ee114-6c61-4775-b8d1-e481700bcf17 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:12.104497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ee114-6c61-4775-b8d1-e481700bcf17 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:12.104774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8ee114-6c61-4775-b8d1-e481700bcf17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:12.114564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-36eaeeb8-422c-4e69-b3a3-668aa36da385 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:49:12.114564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:12.114564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:12.114564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:12.114564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:12.114564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:12.114564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:12.114564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:12.114564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:12.114564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:12.114564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:12.114564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:12.114564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:49:12.114564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:49:12.114564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:49:12.115439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:49:12.115439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:12.115439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:12.115439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:12.115439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:12.115439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:12.115439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:12.115439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:12.115439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:12.115439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:12.115439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:12.115439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:12.115439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:12.115439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:12.115439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:12.115439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:12.115439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:12.116517 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:49:12.116517 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:49:12.116517 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:49:12.116517 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:12.117098 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8ee114-6c61-4775-b8d1-e481700bcf17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:49:12.117202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 552/1091] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:49:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:49:12.121396 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eaeeb8-422c-4e69-b3a3-668aa36da385 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:12.122159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-3c8ee114-6c61-4775-b8d1-e481700bcf17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:49:12.122159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:12.122159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:12.122159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:12.122159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:12.122159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:12.122159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:12.122159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:12.122159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:12.122159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:12.122159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:12.122159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:12.122159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:49:12.122159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:49:12.122159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:49:12.122159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:49:12.123283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:49:12.123283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:49:12.123283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:49:12.123283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:49:12.123283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:12.123283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:12.123283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:12.123283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:12.123283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:12.123283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:12.123283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:12.123283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:12.123283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:12.123283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:12.123283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:12.123283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:12.124129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:12.124129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:12.124129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:12.124129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:12.124129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:49:12.124129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:49:12.124129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:49:12.124129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:49:12.124129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:49:12.124129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:49:12.124129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:49:12.124129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:49:12.124129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 540/1092] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:49:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:49:12.129432 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ee114-6c61-4775-b8d1-e481700bcf17 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:12.142975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:12.224891 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36eaeeb8-422c-4e69-b3a3-668aa36da385 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:12.232611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8ee114-6c61-4775-b8d1-e481700bcf17 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:12.258496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a50b43-d817-41ab-98b6-eead131d3f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:12.259212 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a50b43-d817-41ab-98b6-eead131d3f40 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:12.259480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a50b43-d817-41ab-98b6-eead131d3f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:12.259758 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a50b43-d817-41ab-98b6-eead131d3f40 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:12.260479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a50b43-d817-41ab-98b6-eead131d3f40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:12.260791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a50b43-d817-41ab-98b6-eead131d3f40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:12.262150 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a50b43-d817-41ab-98b6-eead131d3f40 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:12.266129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fbe14-c1ae-4711-a1c5-bde9029cbdbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:12.266892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fbe14-c1ae-4711-a1c5-bde9029cbdbb None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:12.267183 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fbe14-c1ae-4711-a1c5-bde9029cbdbb None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:12.267467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23fbe14-c1ae-4711-a1c5-bde9029cbdbb None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:12.268207 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fbe14-c1ae-4711-a1c5-bde9029cbdbb None admin] RBAC: Adding query filter params (name=admin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:12.268525 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fbe14-c1ae-4711-a1c5-bde9029cbdbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:12.269886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23fbe14-c1ae-4711-a1c5-bde9029cbdbb None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:12.273894 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 553/1093] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:49:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:49:12.283358 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 541/1094] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:49:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:49:12.283521 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a50b43-d817-41ab-98b6-eead131d3f40 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:12.284319 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fbe14-c1ae-4711-a1c5-bde9029cbdbb None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:12.302173 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:12.350983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:12.378979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a50b43-d817-41ab-98b6-eead131d3f40 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:12.383269 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23fbe14-c1ae-4711-a1c5-bde9029cbdbb None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:12.412435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee56e7-db20-4c18-902d-4bb6d8dff37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:12.413222 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee56e7-db20-4c18-902d-4bb6d8dff37d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:12.413508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee56e7-db20-4c18-902d-4bb6d8dff37d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:12.413784 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaee56e7-db20-4c18-902d-4bb6d8dff37d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:12.415733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d895fc4-7bf5-4e03-968f-8b041fe5b589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:12.416596 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d895fc4-7bf5-4e03-968f-8b041fe5b589 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:12.416883 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d895fc4-7bf5-4e03-968f-8b041fe5b589 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:12.417158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d895fc4-7bf5-4e03-968f-8b041fe5b589 None admin] PATH_INFO: `/v3/users/95cb7f9094484d55b47b6304995ab1fb` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:12.418333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d895fc4-7bf5-4e03-968f-8b041fe5b589 None admin] RBAC: Authorizing `identity:get_user(user_id=95cb7f9094484d55b47b6304995ab1fb)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:12.419790 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d895fc4-7bf5-4e03-968f-8b041fe5b589 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:12.420658 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee56e7-db20-4c18-902d-4bb6d8dff37d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:12.421442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 542/1095] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:49:12 2022] GET /identity/v3/users/95cb7f9094484d55b47b6304995ab1fb => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:12.422074 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaee56e7-db20-4c18-902d-4bb6d8dff37d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:12.427777 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d895fc4-7bf5-4e03-968f-8b041fe5b589 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:12.436908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-aaee56e7-db20-4c18-902d-4bb6d8dff37d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 22:49:12.436908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:12.436908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:12.436908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:12.436908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:12.436908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:12.436908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:12.436908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:12.436908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:12.436908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:12.436908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:12.436908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:12.436908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:49:12.436908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:49:12.436908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:49:12.436908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:49:12.437994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:12.437994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:12.437994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:49:12.437994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:49:12.437994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:49:12.437994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:49:12.437994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:12.437994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:12.437994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:12.437994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:12.437994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:12.437994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:12.437994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:12.437994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:12.437994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:12.437994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:12.437994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:12.437994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:12.439556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:12.439556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:12.439556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:49:12.439556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:49:12.439556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:49:12.439556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:49:12.439556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:49:12.439556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:49:12.439556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 22:49:12.439556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:12.439556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 554/1096] 10.222.0.17 () {56 vars in 1093 bytes} [Thu Jun 16 22:49:12 2022] GET /identity/v3/users/nova => generated 82 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:49:12.444126 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee56e7-db20-4c18-902d-4bb6d8dff37d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:12.523534 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:12.524768 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d895fc4-7bf5-4e03-968f-8b041fe5b589 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:12.543723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaee56e7-db20-4c18-902d-4bb6d8dff37d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:12.558541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b366c-e00e-4185-9bde-6b2b42289ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:12.559171 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b366c-e00e-4185-9bde-6b2b42289ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:12.559457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b366c-e00e-4185-9bde-6b2b42289ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:12.559733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2b366c-e00e-4185-9bde-6b2b42289ad7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:12.570707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:12.578066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293db66-b2d4-4afe-a998-4e50d12cd128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:12.578764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293db66-b2d4-4afe-a998-4e50d12cd128 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:12.579077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293db66-b2d4-4afe-a998-4e50d12cd128 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:12.579358 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5293db66-b2d4-4afe-a998-4e50d12cd128 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:12.580186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293db66-b2d4-4afe-a998-4e50d12cd128 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:12.580495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293db66-b2d4-4afe-a998-4e50d12cd128 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:12.581914 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5293db66-b2d4-4afe-a998-4e50d12cd128 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:12.598743 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 555/1097] 10.222.0.17 () {56 vars in 1133 bytes} [Thu Jun 16 22:49:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:49:12.604325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293db66-b2d4-4afe-a998-4e50d12cd128 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:12.627750 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b366c-e00e-4185-9bde-6b2b42289ad7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:12.629191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2b366c-e00e-4185-9bde-6b2b42289ad7 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:12.701140 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5293db66-b2d4-4afe-a998-4e50d12cd128 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:12.726746 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 543/1098] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:49:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:49:12.732371 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b366c-e00e-4185-9bde-6b2b42289ad7 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:12.740113 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef523e4d-b1a6-45fb-8178-fcfdd216785d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:12.740772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef523e4d-b1a6-45fb-8178-fcfdd216785d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:12.741064 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef523e4d-b1a6-45fb-8178-fcfdd216785d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:12.741346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef523e4d-b1a6-45fb-8178-fcfdd216785d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:12.751898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:12.776437 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:12.812248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef523e4d-b1a6-45fb-8178-fcfdd216785d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:12.813782 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef523e4d-b1a6-45fb-8178-fcfdd216785d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:12.833926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2b366c-e00e-4185-9bde-6b2b42289ad7 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:12.868610 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a426d8-f0e0-4404-8246-789494127b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:12.869459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a426d8-f0e0-4404-8246-789494127b8a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:12.869741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a426d8-f0e0-4404-8246-789494127b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:12.870014 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a426d8-f0e0-4404-8246-789494127b8a None admin] PATH_INFO: `/v3/projects/6b1cad07ad694f78ad30147bda444ce5` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:12.871194 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a426d8-f0e0-4404-8246-789494127b8a None admin] RBAC: Authorizing `identity:get_project(project_id=6b1cad07ad694f78ad30147bda444ce5)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:12.872611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a426d8-f0e0-4404-8246-789494127b8a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:12.874291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 544/1099] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:49:12 2022] GET /identity/v3/projects/6b1cad07ad694f78ad30147bda444ce5 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:49:12.880199 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a426d8-f0e0-4404-8246-789494127b8a None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:12.917718 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 556/1100] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:49:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 22:49:12.918934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef523e4d-b1a6-45fb-8178-fcfdd216785d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:12.957218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:12.977301 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a426d8-f0e0-4404-8246-789494127b8a None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:12.987614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:13.012131 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78177f-807b-426c-8526-f02ec5bca6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:13.012783 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78177f-807b-426c-8526-f02ec5bca6df None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:13.013069 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78177f-807b-426c-8526-f02ec5bca6df None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:13.013345 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef78177f-807b-426c-8526-f02ec5bca6df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:13.014165 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78177f-807b-426c-8526-f02ec5bca6df None admin] RBAC: Adding query filter params (role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.project.id=6b1cad07ad694f78ad30147bda444ce5, user.id=95cb7f9094484d55b47b6304995ab1fb) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:13.014477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78177f-807b-426c-8526-f02ec5bca6df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:13.015939 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef78177f-807b-426c-8526-f02ec5bca6df None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:13.016572 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef523e4d-b1a6-45fb-8178-fcfdd216785d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:13.022604 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 545/1101] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:49:12 2022] GET /identity/v3/role_assignments?user.id=95cb7f9094484d55b47b6304995ab1fb&scope.project.id=6b1cad07ad694f78ad30147bda444ce5&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:49:13.058635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94d1a2-fb47-43ba-bb78-11f8ef09e252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:13.059454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94d1a2-fb47-43ba-bb78-11f8ef09e252 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:13.059738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94d1a2-fb47-43ba-bb78-11f8ef09e252 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:13.060013 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c94d1a2-fb47-43ba-bb78-11f8ef09e252 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:13.071749 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c94d1a2-fb47-43ba-bb78-11f8ef09e252 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:13.073134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c94d1a2-fb47-43ba-bb78-11f8ef09e252 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:13.092654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-2c94d1a2-fb47-43ba-bb78-11f8ef09e252 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:49:13.092654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:13.092654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:13.092654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:13.092654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:13.092654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:13.092654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:13.092654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:13.092654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:13.092654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:13.092654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:13.092654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:13.092654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:49:13.092654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:49:13.092654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:49:13.093836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:49:13.093836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:13.093836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:13.093836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:13.093836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:13.093836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:13.093836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:13.093836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:13.093836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:13.093836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:13.093836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:13.093836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:13.093836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:13.093836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:13.093836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:13.093836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:13.093836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:13.095308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:49:13.095308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:49:13.095308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:49:13.095308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:13.095308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 557/1102] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:49:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:49:13.100455 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78177f-807b-426c-8526-f02ec5bca6df None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:13.203278 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef78177f-807b-426c-8526-f02ec5bca6df None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:13.238678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54362851-f76c-47df-ae81-5a2ab3457298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:13.239342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54362851-f76c-47df-ae81-5a2ab3457298 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:13.239633 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54362851-f76c-47df-ae81-5a2ab3457298 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:13.239924 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54362851-f76c-47df-ae81-5a2ab3457298 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:13.240643 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54362851-f76c-47df-ae81-5a2ab3457298 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:13.240957 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54362851-f76c-47df-ae81-5a2ab3457298 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:13.242322 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54362851-f76c-47df-ae81-5a2ab3457298 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:13.253617 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 546/1103] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:13.259348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94d1a2-fb47-43ba-bb78-11f8ef09e252 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:13.348617 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:13.358746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c94d1a2-fb47-43ba-bb78-11f8ef09e252 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:13.393495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb0459-7d3e-4a63-9e51-be9584195fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:13.394292 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb0459-7d3e-4a63-9e51-be9584195fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:13.394581 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb0459-7d3e-4a63-9e51-be9584195fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:13.394893 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cb0459-7d3e-4a63-9e51-be9584195fb0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:13.401796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb0459-7d3e-4a63-9e51-be9584195fb0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:13.403263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cb0459-7d3e-4a63-9e51-be9584195fb0 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:13.416834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-d8cb0459-7d3e-4a63-9e51-be9584195fb0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:49:13.416834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:13.416834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:13.416834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:13.416834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:13.416834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:13.416834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:13.416834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:13.416834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:13.416834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:13.416834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:13.416834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:13.416834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:49:13.416834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:49:13.416834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:49:13.417824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:49:13.417824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:13.417824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:13.417824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:13.417824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:13.417824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:13.417824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:13.417824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:13.417824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:13.417824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:13.417824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:13.417824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:13.417824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:13.417824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:13.417824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:13.417824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:13.417824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:13.418741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:49:13.418741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:49:13.418741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:49:13.418741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:49:13.418741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:49:13.418741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:49:13.418741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:49:13.418741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:13.418741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 558/1104] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:49:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:49:13.424261 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54362851-f76c-47df-ae81-5a2ab3457298 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:13.455468 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:13.522838 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54362851-f76c-47df-ae81-5a2ab3457298 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:13.556422 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda3d4fc-9ea2-4026-b3d9-b48d195a9c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:13.557087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3d4fc-9ea2-4026-b3d9-b48d195a9c86 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:13.557381 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3d4fc-9ea2-4026-b3d9-b48d195a9c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:13.557663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda3d4fc-9ea2-4026-b3d9-b48d195a9c86 None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:13.558439 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3d4fc-9ea2-4026-b3d9-b48d195a9c86 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:13.558785 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3d4fc-9ea2-4026-b3d9-b48d195a9c86 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:13.560214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda3d4fc-9ea2-4026-b3d9-b48d195a9c86 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:13.571993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 547/1105] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:49:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:13.577851 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb0459-7d3e-4a63-9e51-be9584195fb0 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:13.675458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cb0459-7d3e-4a63-9e51-be9584195fb0 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:13.708293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3740756-8635-44a8-bf26-77489b2ba8f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:13.708943 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3740756-8635-44a8-bf26-77489b2ba8f4 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:13.709228 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3740756-8635-44a8-bf26-77489b2ba8f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:13.709506 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3740756-8635-44a8-bf26-77489b2ba8f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:13.710333 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3740756-8635-44a8-bf26-77489b2ba8f4 None admin] RBAC: Adding query filter params (role.id=c6bedc2399d04c78a942f96283ed4560, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=4180f2593149437fa27effd8d1338d10) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:13.710680 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3740756-8635-44a8-bf26-77489b2ba8f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:13.712103 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3740756-8635-44a8-bf26-77489b2ba8f4 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:13.717951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 559/1106] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:49:13 2022] GET /identity/v3/role_assignments?user.id=4180f2593149437fa27effd8d1338d10&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=c6bedc2399d04c78a942f96283ed4560 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:15.175899 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1ae98-fb77-4ce3-94d4-9915b797a892 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:15.176527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1ae98-fb77-4ce3-94d4-9915b797a892 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:15.176527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba1ae98-fb77-4ce3-94d4-9915b797a892 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:15.177803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 548/1107] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:15.182908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe81f1-b47f-4f67-bf48-bf4fcd7bc42f None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:15.183302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe81f1-b47f-4f67-bf48-bf4fcd7bc42f None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:15.183477 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fe81f1-b47f-4f67-bf48-bf4fcd7bc42f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:15.200260 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-f7fe81f1-b47f-4f67-bf48-bf4fcd7bc42f None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:15.201441 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-f7fe81f1-b47f-4f67-bf48-bf4fcd7bc42f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:15.211008 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:15.267368 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 560/1108] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:15.291818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1ae98-fb77-4ce3-94d4-9915b797a892 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:15.379712 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:15.389178 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba1ae98-fb77-4ce3-94d4-9915b797a892 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:15.422281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2df970-b2c1-4635-bcb6-438892fbd3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:15.423595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2df970-b2c1-4635-bcb6-438892fbd3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:15.423888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2df970-b2c1-4635-bcb6-438892fbd3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:15.424167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2df970-b2c1-4635-bcb6-438892fbd3d5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:15.425334 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2df970-b2c1-4635-bcb6-438892fbd3d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:15.426835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2df970-b2c1-4635-bcb6-438892fbd3d5 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:15.428393 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 549/1109] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:49:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:15.434008 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe81f1-b47f-4f67-bf48-bf4fcd7bc42f None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:15.529137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fe81f1-b47f-4f67-bf48-bf4fcd7bc42f None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:15.561565 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498fd0b-9171-49e8-8c3e-f4b8aea18684 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:15.562218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498fd0b-9171-49e8-8c3e-f4b8aea18684 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:15.562504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498fd0b-9171-49e8-8c3e-f4b8aea18684 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:15.562942 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498fd0b-9171-49e8-8c3e-f4b8aea18684 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:15.564225 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498fd0b-9171-49e8-8c3e-f4b8aea18684 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:15.565626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498fd0b-9171-49e8-8c3e-f4b8aea18684 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:15.580913 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:15.586668 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 561/1110] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 22:49:15 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:49:17.707700 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc38899-c70f-4bd9-a97f-824efa41c147 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:17.708263 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc38899-c70f-4bd9-a97f-824efa41c147 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:17.708263 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc38899-c70f-4bd9-a97f-824efa41c147 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:17.709622 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 550/1111] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:17.714793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b5990-518b-44f2-acb2-dd0fa54afa14 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:17.715223 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b5990-518b-44f2-acb2-dd0fa54afa14 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:17.715359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b5990-518b-44f2-acb2-dd0fa54afa14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:17.732159 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-732b5990-518b-44f2-acb2-dd0fa54afa14 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:17.733323 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-732b5990-518b-44f2-acb2-dd0fa54afa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:17.796760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 562/1112] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:17.819823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc38899-c70f-4bd9-a97f-824efa41c147 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:17.918107 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc38899-c70f-4bd9-a97f-824efa41c147 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:17.950075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718d48b-afdc-46b0-b48d-521bff55a0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:17.950944 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718d48b-afdc-46b0-b48d-521bff55a0fc None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:17.951233 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718d48b-afdc-46b0-b48d-521bff55a0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:17.951510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b718d48b-afdc-46b0-b48d-521bff55a0fc None admin] PATH_INFO: `/v3/roles/bda1c48ca2534bc6b2ca4f3ff479b143` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:17.952960 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718d48b-afdc-46b0-b48d-521bff55a0fc None admin] RBAC: Authorizing `identity:get_role(role_id=bda1c48ca2534bc6b2ca4f3ff479b143)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:17.954879 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b718d48b-afdc-46b0-b48d-521bff55a0fc None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:17.956240 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 551/1113] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:49:17 2022] GET /identity/v3/roles/bda1c48ca2534bc6b2ca4f3ff479b143 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:17.961985 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b5990-518b-44f2-acb2-dd0fa54afa14 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:18.077675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b5990-518b-44f2-acb2-dd0fa54afa14 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:18.105003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:18.110870 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ccf7e-aa46-4ba6-99b1-955303d290b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:18.111741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ccf7e-aa46-4ba6-99b1-955303d290b6 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:18.112024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ccf7e-aa46-4ba6-99b1-955303d290b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:18.112302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14ccf7e-aa46-4ba6-99b1-955303d290b6 None admin] PATH_INFO: `/v3/users/d9db70c9e5274d8a936d1c6f7707649c` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:18.124847 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ccf7e-aa46-4ba6-99b1-955303d290b6 None admin] RBAC: Authorizing `identity:get_user(user_id=d9db70c9e5274d8a936d1c6f7707649c)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:18.126816 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14ccf7e-aa46-4ba6-99b1-955303d290b6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:18.128450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 563/1114] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:49:17 2022] GET /identity/v3/users/d9db70c9e5274d8a936d1c6f7707649c => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:18.134219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718d48b-afdc-46b0-b48d-521bff55a0fc None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:18.161801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:18.231470 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b718d48b-afdc-46b0-b48d-521bff55a0fc None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:18.264060 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fcfd8-c5a6-4ad9-a130-f3f0f75faccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:18.264709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fcfd8-c5a6-4ad9-a130-f3f0f75faccd None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:18.264993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fcfd8-c5a6-4ad9-a130-f3f0f75faccd None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:18.265269 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05fcfd8-c5a6-4ad9-a130-f3f0f75faccd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:18.330834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05fcfd8-c5a6-4ad9-a130-f3f0f75faccd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:18.332267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05fcfd8-c5a6-4ad9-a130-f3f0f75faccd None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:18.357307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:18.426719 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 552/1115] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:49:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:49:18.432829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ccf7e-aa46-4ba6-99b1-955303d290b6 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:18.528983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14ccf7e-aa46-4ba6-99b1-955303d290b6 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:18.561084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dc0082-bbd4-4c8c-94d5-7c52db1c4837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:18.561932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc0082-bbd4-4c8c-94d5-7c52db1c4837 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:18.562213 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc0082-bbd4-4c8c-94d5-7c52db1c4837 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:18.562487 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dc0082-bbd4-4c8c-94d5-7c52db1c4837 None admin] PATH_INFO: `/v3/projects/6b1cad07ad694f78ad30147bda444ce5` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:18.564250 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc0082-bbd4-4c8c-94d5-7c52db1c4837 None admin] RBAC: Authorizing `identity:get_project(project_id=6b1cad07ad694f78ad30147bda444ce5)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:18.565650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dc0082-bbd4-4c8c-94d5-7c52db1c4837 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:18.567324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 564/1116] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:49:18 2022] GET /identity/v3/projects/6b1cad07ad694f78ad30147bda444ce5 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:18.573197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fcfd8-c5a6-4ad9-a130-f3f0f75faccd None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:18.667107 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05fcfd8-c5a6-4ad9-a130-f3f0f75faccd None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:18.691613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:18.699366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8d72fb-95e4-43b1-8b1d-d548ee5044f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:18.700079 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d72fb-95e4-43b1-8b1d-d548ee5044f9 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:18.700364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d72fb-95e4-43b1-8b1d-d548ee5044f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:18.700642 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8d72fb-95e4-43b1-8b1d-d548ee5044f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:18.701453 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d72fb-95e4-43b1-8b1d-d548ee5044f9 None admin] RBAC: Adding query filter params (role.id=bda1c48ca2534bc6b2ca4f3ff479b143, scope.project.id=6b1cad07ad694f78ad30147bda444ce5, user.id=d9db70c9e5274d8a936d1c6f7707649c) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:18.701762 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d72fb-95e4-43b1-8b1d-d548ee5044f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:18.703214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8d72fb-95e4-43b1-8b1d-d548ee5044f9 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:18.708936 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 553/1117] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:49:18 2022] GET /identity/v3/role_assignments?user.id=d9db70c9e5274d8a936d1c6f7707649c&scope.project.id=6b1cad07ad694f78ad30147bda444ce5&role.id=bda1c48ca2534bc6b2ca4f3ff479b143 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:20.799049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e672b0c-4f9c-47e3-966d-221f23de95f1 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:20.799610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e672b0c-4f9c-47e3-966d-221f23de95f1 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:20.799610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e672b0c-4f9c-47e3-966d-221f23de95f1 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:20.800986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 565/1118] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:20.806014 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18e649-5f74-49ea-9032-c2ee7b576674 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:20.806442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18e649-5f74-49ea-9032-c2ee7b576674 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:20.806574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18e649-5f74-49ea-9032-c2ee7b576674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:20.823430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-7f18e649-5f74-49ea-9032-c2ee7b576674 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:20.824613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-7f18e649-5f74-49ea-9032-c2ee7b576674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:20.888482 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 554/1119] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:20.911087 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e672b0c-4f9c-47e3-966d-221f23de95f1 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:20.967102 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:21.010603 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e672b0c-4f9c-47e3-966d-221f23de95f1 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:21.043591 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e850d-3448-4781-a300-4086f385ae5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:21.044444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e850d-3448-4781-a300-4086f385ae5b None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:21.044733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e850d-3448-4781-a300-4086f385ae5b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:21.045014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6e850d-3448-4781-a300-4086f385ae5b None admin] PATH_INFO: `/v3/roles/bda1c48ca2534bc6b2ca4f3ff479b143` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:21.046488 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e850d-3448-4781-a300-4086f385ae5b None admin] RBAC: Authorizing `identity:get_role(role_id=bda1c48ca2534bc6b2ca4f3ff479b143)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:21.047910 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6e850d-3448-4781-a300-4086f385ae5b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:21.049336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 566/1120] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:49:20 2022] GET /identity/v3/roles/bda1c48ca2534bc6b2ca4f3ff479b143 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:21.055172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18e649-5f74-49ea-9032-c2ee7b576674 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:21.152236 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18e649-5f74-49ea-9032-c2ee7b576674 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:21.174474 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:21.186192 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03578c8c-e6ab-469e-a673-8a88d42d0934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:21.187097 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03578c8c-e6ab-469e-a673-8a88d42d0934 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:21.187385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03578c8c-e6ab-469e-a673-8a88d42d0934 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:21.188167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03578c8c-e6ab-469e-a673-8a88d42d0934 None admin] PATH_INFO: `/v3/users/d9db70c9e5274d8a936d1c6f7707649c` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:21.189348 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03578c8c-e6ab-469e-a673-8a88d42d0934 None admin] RBAC: Authorizing `identity:get_user(user_id=d9db70c9e5274d8a936d1c6f7707649c)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:21.190823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03578c8c-e6ab-469e-a673-8a88d42d0934 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:21.192466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 555/1121] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:49:21 2022] GET /identity/v3/users/d9db70c9e5274d8a936d1c6f7707649c => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:21.198120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e850d-3448-4781-a300-4086f385ae5b None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:21.295320 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6e850d-3448-4781-a300-4086f385ae5b None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:21.318385 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:21.329814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67a942f-5ea3-453e-9f18-603edd4c0940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:21.330707 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a942f-5ea3-453e-9f18-603edd4c0940 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:21.330999 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a942f-5ea3-453e-9f18-603edd4c0940 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:21.331279 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67a942f-5ea3-453e-9f18-603edd4c0940 None admin] PATH_INFO: `/v3/projects/6b1cad07ad694f78ad30147bda444ce5` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:21.332459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a942f-5ea3-453e-9f18-603edd4c0940 None admin] RBAC: Authorizing `identity:get_project(project_id=6b1cad07ad694f78ad30147bda444ce5)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:21.333872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67a942f-5ea3-453e-9f18-603edd4c0940 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:21.335595 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 567/1122] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:49:21 2022] GET /identity/v3/projects/6b1cad07ad694f78ad30147bda444ce5 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:21.341572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03578c8c-e6ab-469e-a673-8a88d42d0934 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:21.437146 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03578c8c-e6ab-469e-a673-8a88d42d0934 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:21.469255 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebf8eb8-eb3b-4a62-99f4-426006e5f0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:21.470127 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf8eb8-eb3b-4a62-99f4-426006e5f0b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:21.470410 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf8eb8-eb3b-4a62-99f4-426006e5f0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:21.470714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebf8eb8-eb3b-4a62-99f4-426006e5f0b7 None admin] PATH_INFO: `/v3/projects/6b1cad07ad694f78ad30147bda444ce5/users/d9db70c9e5274d8a936d1c6f7707649c/roles/bda1c48ca2534bc6b2ca4f3ff479b143` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:21.472853 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf8eb8-eb3b-4a62-99f4-426006e5f0b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=6b1cad07ad694f78ad30147bda444ce5, user_id=d9db70c9e5274d8a936d1c6f7707649c, role_id=bda1c48ca2534bc6b2ca4f3ff479b143)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:21.474256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebf8eb8-eb3b-4a62-99f4-426006e5f0b7 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:21.486908 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 556/1123] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:49:21 2022] PUT /identity/v3/projects/6b1cad07ad694f78ad30147bda444ce5/users/d9db70c9e5274d8a936d1c6f7707649c/roles/bda1c48ca2534bc6b2ca4f3ff479b143 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:49:23.563345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946a68c-eca6-473a-b9f8-6e00666c0094 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:23.563861 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946a68c-eca6-473a-b9f8-6e00666c0094 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:23.563861 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946a68c-eca6-473a-b9f8-6e00666c0094 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:23.565174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 568/1124] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:23.570346 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b81227-73db-4f5c-92f0-22f05e5c6e37 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:23.570748 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b81227-73db-4f5c-92f0-22f05e5c6e37 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:23.570941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b81227-73db-4f5c-92f0-22f05e5c6e37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:23.587181 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-66b81227-73db-4f5c-92f0-22f05e5c6e37 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:23.588362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-66b81227-73db-4f5c-92f0-22f05e5c6e37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:23.633996 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:23.655900 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 557/1125] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:23.679630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946a68c-eca6-473a-b9f8-6e00666c0094 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:23.778023 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946a68c-eca6-473a-b9f8-6e00666c0094 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:23.810744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb586ab9-bb72-44d5-8516-7ae2943e2163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:23.811589 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb586ab9-bb72-44d5-8516-7ae2943e2163 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:23.811878 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb586ab9-bb72-44d5-8516-7ae2943e2163 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:23.812158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb586ab9-bb72-44d5-8516-7ae2943e2163 None admin] PATH_INFO: `/v3/roles/bda1c48ca2534bc6b2ca4f3ff479b143` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:23.813698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb586ab9-bb72-44d5-8516-7ae2943e2163 None admin] RBAC: Authorizing `identity:get_role(role_id=bda1c48ca2534bc6b2ca4f3ff479b143)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:23.815044 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb586ab9-bb72-44d5-8516-7ae2943e2163 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:23.816388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 569/1126] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:49:23 2022] GET /identity/v3/roles/bda1c48ca2534bc6b2ca4f3ff479b143 => generated 223 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:23.821879 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b81227-73db-4f5c-92f0-22f05e5c6e37 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:23.917271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b81227-73db-4f5c-92f0-22f05e5c6e37 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:23.949351 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb82f9-0ff9-4ebf-9666-f890ad773215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:23.950200 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb82f9-0ff9-4ebf-9666-f890ad773215 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:23.950490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb82f9-0ff9-4ebf-9666-f890ad773215 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:23.950801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb82f9-0ff9-4ebf-9666-f890ad773215 None admin] PATH_INFO: `/v3/users/d9db70c9e5274d8a936d1c6f7707649c` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:23.951976 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb82f9-0ff9-4ebf-9666-f890ad773215 None admin] RBAC: Authorizing `identity:get_user(user_id=d9db70c9e5274d8a936d1c6f7707649c)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:23.953933 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb82f9-0ff9-4ebf-9666-f890ad773215 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:23.955563 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 558/1127] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:49:23 2022] GET /identity/v3/users/d9db70c9e5274d8a936d1c6f7707649c => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:23.961015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb586ab9-bb72-44d5-8516-7ae2943e2163 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:24.019746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:24.061248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb586ab9-bb72-44d5-8516-7ae2943e2163 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:24.095044 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f590040-796e-47bb-ba63-d2415b160790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:24.095631 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f590040-796e-47bb-ba63-d2415b160790 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:24.095918 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f590040-796e-47bb-ba63-d2415b160790 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:24.096199 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f590040-796e-47bb-ba63-d2415b160790 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:24.161131 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f590040-796e-47bb-ba63-d2415b160790 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:24.162556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f590040-796e-47bb-ba63-d2415b160790 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:24.217755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:24.258705 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 570/1128] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:49:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:49:24.264582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb82f9-0ff9-4ebf-9666-f890ad773215 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:24.316262 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:24.368320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb82f9-0ff9-4ebf-9666-f890ad773215 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:24.401742 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b37f77-c360-4c71-b932-4ed0cf6954e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:24.402584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b37f77-c360-4c71-b932-4ed0cf6954e6 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:24.402895 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b37f77-c360-4c71-b932-4ed0cf6954e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:24.403170 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b37f77-c360-4c71-b932-4ed0cf6954e6 None admin] PATH_INFO: `/v3/projects/6b1cad07ad694f78ad30147bda444ce5` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:24.404320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b37f77-c360-4c71-b932-4ed0cf6954e6 None admin] RBAC: Authorizing `identity:get_project(project_id=6b1cad07ad694f78ad30147bda444ce5)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:24.405730 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b37f77-c360-4c71-b932-4ed0cf6954e6 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:24.407477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 559/1129] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:49:24 2022] GET /identity/v3/projects/6b1cad07ad694f78ad30147bda444ce5 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:24.413573 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f590040-796e-47bb-ba63-d2415b160790 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:24.507828 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f590040-796e-47bb-ba63-d2415b160790 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:24.539770 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cde89fc-4616-4bc1-9ecf-5ff9f46100c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:24.540406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde89fc-4616-4bc1-9ecf-5ff9f46100c2 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:24.540687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde89fc-4616-4bc1-9ecf-5ff9f46100c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:24.540960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cde89fc-4616-4bc1-9ecf-5ff9f46100c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:24.541775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde89fc-4616-4bc1-9ecf-5ff9f46100c2 None admin] RBAC: Adding query filter params (role.id=bda1c48ca2534bc6b2ca4f3ff479b143, scope.project.id=6b1cad07ad694f78ad30147bda444ce5, user.id=d9db70c9e5274d8a936d1c6f7707649c) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:24.542085 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde89fc-4616-4bc1-9ecf-5ff9f46100c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:24.543518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cde89fc-4616-4bc1-9ecf-5ff9f46100c2 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:24.549634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 571/1130] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:49:24 2022] GET /identity/v3/role_assignments?user.id=d9db70c9e5274d8a936d1c6f7707649c&scope.project.id=6b1cad07ad694f78ad30147bda444ce5&role.id=bda1c48ca2534bc6b2ca4f3ff479b143 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:26.642307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374faa4c-c462-4452-a1d8-20d4f9f7d1a2 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:26.642848 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374faa4c-c462-4452-a1d8-20d4f9f7d1a2 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:26.642939 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374faa4c-c462-4452-a1d8-20d4f9f7d1a2 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:26.644279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 560/1131] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:26.649736 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ad6c2-4e59-4666-8854-65aa1bfe4d28 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:26.650149 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ad6c2-4e59-4666-8854-65aa1bfe4d28 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:26.650297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3ad6c2-4e59-4666-8854-65aa1bfe4d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:26.666497 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-fc3ad6c2-4e59-4666-8854-65aa1bfe4d28 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:26.667697 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-fc3ad6c2-4e59-4666-8854-65aa1bfe4d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:26.684850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:26.731807 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 572/1132] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:26.755682 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374faa4c-c462-4452-a1d8-20d4f9f7d1a2 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:26.852343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374faa4c-c462-4452-a1d8-20d4f9f7d1a2 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:26.868350 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:26.886080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e6309e-d909-475c-97ea-3b57461b32ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:26.886880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6309e-d909-475c-97ea-3b57461b32ed None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:26.887169 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6309e-d909-475c-97ea-3b57461b32ed None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:26.887448 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e6309e-d909-475c-97ea-3b57461b32ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:26.888604 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e6309e-d909-475c-97ea-3b57461b32ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:26.890026 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e6309e-d909-475c-97ea-3b57461b32ed None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:26.891646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 561/1133] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:49:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:26.897294 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ad6c2-4e59-4666-8854-65aa1bfe4d28 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:26.992626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3ad6c2-4e59-4666-8854-65aa1bfe4d28 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:27.025883 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c194c1b-968f-4d98-846c-ec093706e516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:27.026534 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c194c1b-968f-4d98-846c-ec093706e516 None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:27.026850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c194c1b-968f-4d98-846c-ec093706e516 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:27.027128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c194c1b-968f-4d98-846c-ec093706e516 None admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:27.027826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c194c1b-968f-4d98-846c-ec093706e516 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:27.029230 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c194c1b-968f-4d98-846c-ec093706e516 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:27.055822 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 573/1134] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 22:49:26 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:49:29.191485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e06bfb-3219-4934-a85c-60f359a9e51a None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:29.192034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e06bfb-3219-4934-a85c-60f359a9e51a None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:29.192034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e06bfb-3219-4934-a85c-60f359a9e51a None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:29.193402 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 562/1135] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:29.198609 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d169dd-eb64-4475-b2c4-27e1fa8b80be None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:29.199031 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d169dd-eb64-4475-b2c4-27e1fa8b80be None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:29.199168 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d169dd-eb64-4475-b2c4-27e1fa8b80be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:29.210661 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:29.216509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-d5d169dd-eb64-4475-b2c4-27e1fa8b80be None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:29.218197 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-d5d169dd-eb64-4475-b2c4-27e1fa8b80be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:29.283676 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 574/1136] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:29.308266 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e06bfb-3219-4934-a85c-60f359a9e51a None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:29.405105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e06bfb-3219-4934-a85c-60f359a9e51a None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:29.448656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946d143b-aa8f-4113-9631-9934c3fba141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:29.449368 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d143b-aa8f-4113-9631-9934c3fba141 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:29.449654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d143b-aa8f-4113-9631-9934c3fba141 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:29.449930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946d143b-aa8f-4113-9631-9934c3fba141 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:29.451122 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d143b-aa8f-4113-9631-9934c3fba141 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:29.452567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946d143b-aa8f-4113-9631-9934c3fba141 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:29.454168 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 563/1137] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:49:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:29.459874 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d169dd-eb64-4475-b2c4-27e1fa8b80be None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:29.556639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d169dd-eb64-4475-b2c4-27e1fa8b80be None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:29.588616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb4713b2-d4bd-4814-8786-15e5b95b52ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:29.589275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4713b2-d4bd-4814-8786-15e5b95b52ea None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:29.589564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4713b2-d4bd-4814-8786-15e5b95b52ea None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:29.589843 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb4713b2-d4bd-4814-8786-15e5b95b52ea None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:29.590556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4713b2-d4bd-4814-8786-15e5b95b52ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:29.592005 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb4713b2-d4bd-4814-8786-15e5b95b52ea None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:29.599858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:29.612830 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 575/1138] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 22:49:29 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:49:31.757081 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41fbbb-2504-45e6-9341-4edf2a320cf3 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:31.757604 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41fbbb-2504-45e6-9341-4edf2a320cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:31.757694 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b41fbbb-2504-45e6-9341-4edf2a320cf3 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:31.759047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 564/1139] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:31.764191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6fdf3-3fd3-4bfd-a715-37391ba59f9a None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:31.764600 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6fdf3-3fd3-4bfd-a715-37391ba59f9a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:31.765305 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6fdf3-3fd3-4bfd-a715-37391ba59f9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:31.781850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-6fb6fdf3-3fd3-4bfd-a715-37391ba59f9a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:31.783058 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-6fb6fdf3-3fd3-4bfd-a715-37391ba59f9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:31.847884 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 576/1140] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:31.870431 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41fbbb-2504-45e6-9341-4edf2a320cf3 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:31.923556 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:31.969066 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b41fbbb-2504-45e6-9341-4edf2a320cf3 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:32.001968 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e56e74-5916-49ce-9e6c-35b8cc7fab25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:32.002775 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e56e74-5916-49ce-9e6c-35b8cc7fab25 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:32.003066 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e56e74-5916-49ce-9e6c-35b8cc7fab25 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:32.003348 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e56e74-5916-49ce-9e6c-35b8cc7fab25 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:32.013436 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e56e74-5916-49ce-9e6c-35b8cc7fab25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:49:32.021059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-88e56e74-5916-49ce-9e6c-35b8cc7fab25 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:49:32.021059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:32.021059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:32.021059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:32.021059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:32.021059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:32.021059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:32.021059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:32.021059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:32.021059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:32.021059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:32.021059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:32.021059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:49:32.021059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:49:32.021059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:49:32.021059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:49:32.022214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:49:32.022214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:49:32.022214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:49:32.022214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:49:32.022214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:32.022214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:32.022214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:32.022214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:32.022214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:32.022214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:32.022214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:32.022214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:32.022214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:32.022214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:32.022214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:32.022214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:32.023188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:32.023188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:32.023188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:32.023188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:32.023188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:49:32.023188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:49:32.023188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:49:32.023188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:49:32.023188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:49:32.023188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:49:32.023188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:49:32.023188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:49:32.023188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 565/1141] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:49:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:49:32.028367 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6fdf3-3fd3-4bfd-a715-37391ba59f9a None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:32.123744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6fdf3-3fd3-4bfd-a715-37391ba59f9a None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:32.145917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:32.158522 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5811e-4198-4685-aa75-2bd29de5709d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:32.159231 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5811e-4198-4685-aa75-2bd29de5709d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:32.159520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5811e-4198-4685-aa75-2bd29de5709d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:32.159798 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef5811e-4198-4685-aa75-2bd29de5709d None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:32.160544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5811e-4198-4685-aa75-2bd29de5709d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:32.160857 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5811e-4198-4685-aa75-2bd29de5709d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:32.162215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef5811e-4198-4685-aa75-2bd29de5709d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:32.170119 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 577/1142] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:49:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:32.176381 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e56e74-5916-49ce-9e6c-35b8cc7fab25 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:32.273738 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e56e74-5916-49ce-9e6c-35b8cc7fab25 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:32.286436 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:32.307168 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6858f-f6c5-47d7-bc63-e663c7294547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:32.308024 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6858f-f6c5-47d7-bc63-e663c7294547 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:32.308304 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6858f-f6c5-47d7-bc63-e663c7294547 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:32.308575 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb6858f-f6c5-47d7-bc63-e663c7294547 None admin] PATH_INFO: `/v3/users/64b624bd30bd4e9f9f4fda7a2ae79b1f` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:32.321191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6858f-f6c5-47d7-bc63-e663c7294547 None admin] RBAC: Authorizing `identity:get_user(user_id=64b624bd30bd4e9f9f4fda7a2ae79b1f)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:32.322641 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb6858f-f6c5-47d7-bc63-e663c7294547 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:32.324260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 566/1143] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:49:32 2022] GET /identity/v3/users/64b624bd30bd4e9f9f4fda7a2ae79b1f => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:32.329979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5811e-4198-4685-aa75-2bd29de5709d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:32.427044 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef5811e-4198-4685-aa75-2bd29de5709d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:32.458496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80532251-6a10-42ec-927d-9890cec57edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:32.459162 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80532251-6a10-42ec-927d-9890cec57edb None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:32.459452 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80532251-6a10-42ec-927d-9890cec57edb None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:32.459734 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80532251-6a10-42ec-927d-9890cec57edb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:32.509138 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:32.525003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80532251-6a10-42ec-927d-9890cec57edb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:32.526359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80532251-6a10-42ec-927d-9890cec57edb None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:32.623531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 578/1144] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:49:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:49:32.629455 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6858f-f6c5-47d7-bc63-e663c7294547 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:32.725471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb6858f-f6c5-47d7-bc63-e663c7294547 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:32.757626 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baa653a-3c4f-4e09-b0b1-ac2ae7833da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:32.758468 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa653a-3c4f-4e09-b0b1-ac2ae7833da8 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:32.758783 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa653a-3c4f-4e09-b0b1-ac2ae7833da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:32.759064 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baa653a-3c4f-4e09-b0b1-ac2ae7833da8 None admin] PATH_INFO: `/v3/projects/f7fd641915554d4897c037204acf0b7d` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:32.760232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa653a-3c4f-4e09-b0b1-ac2ae7833da8 None admin] RBAC: Authorizing `identity:get_project(project_id=f7fd641915554d4897c037204acf0b7d)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:32.761641 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5baa653a-3c4f-4e09-b0b1-ac2ae7833da8 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:32.763329 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 567/1145] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:49:32 2022] GET /identity/v3/projects/f7fd641915554d4897c037204acf0b7d => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:32.769590 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80532251-6a10-42ec-927d-9890cec57edb None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:32.850311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:32.865746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80532251-6a10-42ec-927d-9890cec57edb None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:32.898209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8552dc3c-e606-4317-82b9-ad8aae46e6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:32.899120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552dc3c-e606-4317-82b9-ad8aae46e6fe None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:32.899415 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552dc3c-e606-4317-82b9-ad8aae46e6fe None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:32.899693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8552dc3c-e606-4317-82b9-ad8aae46e6fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:32.900519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552dc3c-e606-4317-82b9-ad8aae46e6fe None admin] RBAC: Adding query filter params (role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.project.id=f7fd641915554d4897c037204acf0b7d, user.id=64b624bd30bd4e9f9f4fda7a2ae79b1f) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:32.900836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552dc3c-e606-4317-82b9-ad8aae46e6fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:32.902242 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8552dc3c-e606-4317-82b9-ad8aae46e6fe None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:32.908032 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 579/1146] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:49:32 2022] GET /identity/v3/role_assignments?user.id=64b624bd30bd4e9f9f4fda7a2ae79b1f&scope.project.id=f7fd641915554d4897c037204acf0b7d&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:34.983513 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208c9e1-643a-4e79-adf5-0b9e3cf6f7b7 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:34.984079 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208c9e1-643a-4e79-adf5-0b9e3cf6f7b7 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:34.984174 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9208c9e1-643a-4e79-adf5-0b9e3cf6f7b7 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:34.985528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 568/1147] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:34.991389 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f213c2b-0359-4f9d-96ec-eb3422065d87 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:34.991824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f213c2b-0359-4f9d-96ec-eb3422065d87 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:34.991929 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f213c2b-0359-4f9d-96ec-eb3422065d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:35.009059 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-7f213c2b-0359-4f9d-96ec-eb3422065d87 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:35.010225 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7f213c2b-0359-4f9d-96ec-eb3422065d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:35.074487 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 580/1148] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:35.097004 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208c9e1-643a-4e79-adf5-0b9e3cf6f7b7 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:35.137061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:35.198776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9208c9e1-643a-4e79-adf5-0b9e3cf6f7b7 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:35.231091 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf858a-147b-47e4-a6f9-69922a0f5170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:35.231880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf858a-147b-47e4-a6f9-69922a0f5170 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:35.232163 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf858a-147b-47e4-a6f9-69922a0f5170 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:35.232439 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf858a-147b-47e4-a6f9-69922a0f5170 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:35.242796 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcf858a-147b-47e4-a6f9-69922a0f5170 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:49:35.250892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-0bcf858a-147b-47e4-a6f9-69922a0f5170 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:49:35.250892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:35.250892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:35.250892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:35.250892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:35.250892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:35.250892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:35.250892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:35.250892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:35.250892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:35.250892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:35.250892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:35.250892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:49:35.250892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:49:35.250892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:49:35.250892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:49:35.251979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:49:35.251979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:49:35.251979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:49:35.251979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:49:35.251979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:35.251979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:35.251979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:35.251979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:35.251979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:35.251979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:35.251979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:35.251979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:35.251979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:35.251979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:35.251979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:35.251979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:35.253049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:35.253049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:35.253049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:35.253049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:35.253049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:49:35.253049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:49:35.253049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:49:35.253049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:49:35.253049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:49:35.253049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:49:35.253049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:49:35.253049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:49:35.253049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 569/1149] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:49:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:49:35.258106 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f213c2b-0359-4f9d-96ec-eb3422065d87 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:35.332651 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:35.353840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f213c2b-0359-4f9d-96ec-eb3422065d87 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:35.386379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29aab40-287a-44e5-a085-a6d408541ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:35.387063 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29aab40-287a-44e5-a085-a6d408541ade None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:35.387348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29aab40-287a-44e5-a085-a6d408541ade None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:35.387624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29aab40-287a-44e5-a085-a6d408541ade None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:35.388357 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29aab40-287a-44e5-a085-a6d408541ade None admin] RBAC: Adding query filter params (name=admin) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:35.388664 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29aab40-287a-44e5-a085-a6d408541ade None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:35.390048 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29aab40-287a-44e5-a085-a6d408541ade None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:35.397702 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 581/1150] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:49:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:35.403530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf858a-147b-47e4-a6f9-69922a0f5170 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:35.495825 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:35.499860 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf858a-147b-47e4-a6f9-69922a0f5170 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:35.532291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36655d53-65b2-429e-972d-f6bf8ade802f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:35.533145 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36655d53-65b2-429e-972d-f6bf8ade802f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:35.533425 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36655d53-65b2-429e-972d-f6bf8ade802f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:35.533694 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36655d53-65b2-429e-972d-f6bf8ade802f None admin] PATH_INFO: `/v3/users/64b624bd30bd4e9f9f4fda7a2ae79b1f` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:35.534885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36655d53-65b2-429e-972d-f6bf8ade802f None admin] RBAC: Authorizing `identity:get_user(user_id=64b624bd30bd4e9f9f4fda7a2ae79b1f)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:35.536291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36655d53-65b2-429e-972d-f6bf8ade802f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:35.537880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 570/1151] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:49:35 2022] GET /identity/v3/users/64b624bd30bd4e9f9f4fda7a2ae79b1f => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:35.543381 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29aab40-287a-44e5-a085-a6d408541ade None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:35.637976 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29aab40-287a-44e5-a085-a6d408541ade None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:35.670878 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ade031-dfa3-4686-866b-809dc89014dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:35.671722 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ade031-dfa3-4686-866b-809dc89014dc None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:35.672005 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ade031-dfa3-4686-866b-809dc89014dc None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:35.672280 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ade031-dfa3-4686-866b-809dc89014dc None admin] PATH_INFO: `/v3/projects/f7fd641915554d4897c037204acf0b7d` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:35.673435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ade031-dfa3-4686-866b-809dc89014dc None admin] RBAC: Authorizing `identity:get_project(project_id=f7fd641915554d4897c037204acf0b7d)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:35.674864 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ade031-dfa3-4686-866b-809dc89014dc None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:35.676507 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 582/1152] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:49:35 2022] GET /identity/v3/projects/f7fd641915554d4897c037204acf0b7d => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:35.682456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36655d53-65b2-429e-972d-f6bf8ade802f None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:35.778195 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36655d53-65b2-429e-972d-f6bf8ade802f None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:35.809850 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035e2172-1c5e-4542-bc55-cd0da6f68fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:35.810750 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e2172-1c5e-4542-bc55-cd0da6f68fa3 None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:35.811574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e2172-1c5e-4542-bc55-cd0da6f68fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:35.811859 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035e2172-1c5e-4542-bc55-cd0da6f68fa3 None admin] PATH_INFO: `/v3/projects/f7fd641915554d4897c037204acf0b7d/users/64b624bd30bd4e9f9f4fda7a2ae79b1f/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:35.813771 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e2172-1c5e-4542-bc55-cd0da6f68fa3 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7fd641915554d4897c037204acf0b7d, user_id=64b624bd30bd4e9f9f4fda7a2ae79b1f, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:35.815207 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035e2172-1c5e-4542-bc55-cd0da6f68fa3 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:35.828491 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 571/1153] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:49:35 2022] PUT /identity/v3/projects/f7fd641915554d4897c037204acf0b7d/users/64b624bd30bd4e9f9f4fda7a2ae79b1f/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:49:37.899014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15a53f-9a16-40a9-831d-5e3e20cde90c None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:37.899560 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15a53f-9a16-40a9-831d-5e3e20cde90c None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:37.899560 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e15a53f-9a16-40a9-831d-5e3e20cde90c None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:37.900933 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 583/1154] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:37.906726 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abf87c-bb0b-406b-b2a4-50a303d7bd5e None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:37.907122 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abf87c-bb0b-406b-b2a4-50a303d7bd5e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:37.907305 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abf87c-bb0b-406b-b2a4-50a303d7bd5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:37.923676 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-98abf87c-bb0b-406b-b2a4-50a303d7bd5e None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:37.924860 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-98abf87c-bb0b-406b-b2a4-50a303d7bd5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:37.944212 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:37.991308 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 572/1155] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:38.015142 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15a53f-9a16-40a9-831d-5e3e20cde90c None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:38.026468 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:38.114775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e15a53f-9a16-40a9-831d-5e3e20cde90c None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:38.147433 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef32b09-5f99-435f-a80d-74b7d2aaa51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:38.148212 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef32b09-5f99-435f-a80d-74b7d2aaa51f None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:38.148752 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef32b09-5f99-435f-a80d-74b7d2aaa51f None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:38.149078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef32b09-5f99-435f-a80d-74b7d2aaa51f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:38.159796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef32b09-5f99-435f-a80d-74b7d2aaa51f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:49:38.167255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-fef32b09-5f99-435f-a80d-74b7d2aaa51f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:49:38.167255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:38.167255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:38.167255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:38.167255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:38.167255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:38.167255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:38.167255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:38.167255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:38.167255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:38.167255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:38.167255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:38.167255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:49:38.167255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:49:38.167255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:49:38.167255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:49:38.168311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:49:38.168311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:49:38.168311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:49:38.168311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:49:38.168311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:38.168311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:38.168311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:38.168311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:38.168311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:38.168311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:38.168311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:38.168311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:38.168311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:38.168311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:38.168311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:38.168311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:38.169174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:38.169174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:38.169174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:38.169174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:38.169174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:49:38.169174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:49:38.169174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:49:38.169174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:49:38.169174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:49:38.169174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:49:38.169174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:49:38.169174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:38.169174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 584/1156] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:49:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:49:38.174351 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98abf87c-bb0b-406b-b2a4-50a303d7bd5e None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:38.271445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98abf87c-bb0b-406b-b2a4-50a303d7bd5e None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:38.303850 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464bf12-5d72-40dc-b9a5-c04cf9fdfea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:38.304515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464bf12-5d72-40dc-b9a5-c04cf9fdfea7 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:38.304806 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464bf12-5d72-40dc-b9a5-c04cf9fdfea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:38.305091 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c464bf12-5d72-40dc-b9a5-c04cf9fdfea7 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:38.305830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464bf12-5d72-40dc-b9a5-c04cf9fdfea7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:38.306144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464bf12-5d72-40dc-b9a5-c04cf9fdfea7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:38.307548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c464bf12-5d72-40dc-b9a5-c04cf9fdfea7 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:38.315938 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 573/1157] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:49:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:38.321779 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef32b09-5f99-435f-a80d-74b7d2aaa51f None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:38.392566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:38.424244 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef32b09-5f99-435f-a80d-74b7d2aaa51f None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:38.457164 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37843752-80d9-49d4-ab0f-f27d16e4caf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:38.458023 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37843752-80d9-49d4-ab0f-f27d16e4caf7 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:38.458309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37843752-80d9-49d4-ab0f-f27d16e4caf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:38.458583 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37843752-80d9-49d4-ab0f-f27d16e4caf7 None admin] PATH_INFO: `/v3/users/64b624bd30bd4e9f9f4fda7a2ae79b1f` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:38.459781 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37843752-80d9-49d4-ab0f-f27d16e4caf7 None admin] RBAC: Authorizing `identity:get_user(user_id=64b624bd30bd4e9f9f4fda7a2ae79b1f)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:38.461199 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37843752-80d9-49d4-ab0f-f27d16e4caf7 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:38.462849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 585/1158] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:49:38 2022] GET /identity/v3/users/64b624bd30bd4e9f9f4fda7a2ae79b1f => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:38.468690 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464bf12-5d72-40dc-b9a5-c04cf9fdfea7 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:38.487658 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:38.565410 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c464bf12-5d72-40dc-b9a5-c04cf9fdfea7 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:38.597282 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bc183-61ac-4098-b819-e6a05be6b363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:38.597919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bc183-61ac-4098-b819-e6a05be6b363 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:38.598202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bc183-61ac-4098-b819-e6a05be6b363 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:38.598476 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73bc183-61ac-4098-b819-e6a05be6b363 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:38.662010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73bc183-61ac-4098-b819-e6a05be6b363 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:38.663469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73bc183-61ac-4098-b819-e6a05be6b363 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:38.680500 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:38.757312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 574/1159] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:49:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:49:38.763212 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37843752-80d9-49d4-ab0f-f27d16e4caf7 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:38.860833 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37843752-80d9-49d4-ab0f-f27d16e4caf7 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:38.893330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f569038a-4468-4ea4-8058-4e5f432890bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:38.894180 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569038a-4468-4ea4-8058-4e5f432890bf None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:38.894468 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569038a-4468-4ea4-8058-4e5f432890bf None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:38.894775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f569038a-4468-4ea4-8058-4e5f432890bf None admin] PATH_INFO: `/v3/projects/f7fd641915554d4897c037204acf0b7d` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:38.895937 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569038a-4468-4ea4-8058-4e5f432890bf None admin] RBAC: Authorizing `identity:get_project(project_id=f7fd641915554d4897c037204acf0b7d)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:38.897344 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f569038a-4468-4ea4-8058-4e5f432890bf None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:38.899308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 586/1160] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:49:38 2022] GET /identity/v3/projects/f7fd641915554d4897c037204acf0b7d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:38.905989 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bc183-61ac-4098-b819-e6a05be6b363 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:38.999587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73bc183-61ac-4098-b819-e6a05be6b363 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:39.016291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:39.033256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5120612c-f730-4fc2-9111-89ff928e8889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:39.033901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120612c-f730-4fc2-9111-89ff928e8889 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:39.034184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120612c-f730-4fc2-9111-89ff928e8889 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:39.034461 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5120612c-f730-4fc2-9111-89ff928e8889 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:39.035303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120612c-f730-4fc2-9111-89ff928e8889 None admin] RBAC: Adding query filter params (role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.project.id=f7fd641915554d4897c037204acf0b7d, user.id=64b624bd30bd4e9f9f4fda7a2ae79b1f) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:39.035616 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120612c-f730-4fc2-9111-89ff928e8889 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:39.037023 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5120612c-f730-4fc2-9111-89ff928e8889 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:39.042574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 575/1161] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:49:38 2022] GET /identity/v3/role_assignments?user.id=64b624bd30bd4e9f9f4fda7a2ae79b1f&scope.project.id=f7fd641915554d4897c037204acf0b7d&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:41.176017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff42f08-82e2-4f5e-a213-66d89bc32b77 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:41.176577 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff42f08-82e2-4f5e-a213-66d89bc32b77 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:41.176577 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff42f08-82e2-4f5e-a213-66d89bc32b77 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:41.177932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 587/1162] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:41.183320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682469c-6edf-4e74-91f4-ff6db5798030 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:41.183766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682469c-6edf-4e74-91f4-ff6db5798030 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:41.183875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6682469c-6edf-4e74-91f4-ff6db5798030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:41.204288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-6682469c-6edf-4e74-91f4-ff6db5798030 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:41.205457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6682469c-6edf-4e74-91f4-ff6db5798030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:41.268987 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 576/1163] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:41.291726 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff42f08-82e2-4f5e-a213-66d89bc32b77 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:41.296026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:41.390159 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff42f08-82e2-4f5e-a213-66d89bc32b77 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:41.424258 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16ad22-c262-4e24-8161-e732aad5b583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:41.424883 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16ad22-c262-4e24-8161-e732aad5b583 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:41.425166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16ad22-c262-4e24-8161-e732aad5b583 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:41.425440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e16ad22-c262-4e24-8161-e732aad5b583 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:41.497133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16ad22-c262-4e24-8161-e732aad5b583 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:41.499186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e16ad22-c262-4e24-8161-e732aad5b583 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:41.501765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:41.593485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 588/1164] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:49:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:49:41.599357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682469c-6edf-4e74-91f4-ff6db5798030 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:41.693004 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6682469c-6edf-4e74-91f4-ff6db5798030 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:41.706465 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:41.726312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fba39c-5c8d-4a4a-a3c0-2bdf14ebbb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:41.727136 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fba39c-5c8d-4a4a-a3c0-2bdf14ebbb68 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:41.727418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fba39c-5c8d-4a4a-a3c0-2bdf14ebbb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:41.727687 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fba39c-5c8d-4a4a-a3c0-2bdf14ebbb68 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:41.734605 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fba39c-5c8d-4a4a-a3c0-2bdf14ebbb68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:41.736033 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fba39c-5c8d-4a4a-a3c0-2bdf14ebbb68 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:41.749609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-60fba39c-5c8d-4a4a-a3c0-2bdf14ebbb68 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 22:49:41.749609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:41.749609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 16 22:49:41.749609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 16 22:49:41.749609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:49:41.749609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:49:41.749609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:49:41.749609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:49:41.749609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 16 22:49:41.749609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:49:41.749609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:49:41.749609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:49:41.749609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:41.749609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:41.749609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:41.749609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:41.749609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:41.750801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:41.750801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:41.750801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:41.750801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:41.750801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:41.750801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:41.750801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:49:41.750801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:49:41.750801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:49:41.750801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:49:41.750801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:41.750801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:41.750801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:41.750801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:41.750801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:41.750801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:41.751834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:41.751834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:41.751834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:41.751834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:41.751834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:41.751834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:41.751834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:41.751834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:41.751834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:41.751834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:41.751834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 16 22:49:41.751834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:49:41.751834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 22:49:41.751834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:49:41.751834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 577/1165] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 22:49:41 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:49:41.756462 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16ad22-c262-4e24-8161-e732aad5b583 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:41.851654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e16ad22-c262-4e24-8161-e732aad5b583 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:41.853882 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:41.884151 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120360b3-f1bf-4e54-9608-91f304dc1a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:41.884802 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120360b3-f1bf-4e54-9608-91f304dc1a94 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:41.885088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120360b3-f1bf-4e54-9608-91f304dc1a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:41.885367 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120360b3-f1bf-4e54-9608-91f304dc1a94 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:41.886081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120360b3-f1bf-4e54-9608-91f304dc1a94 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:41.886394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120360b3-f1bf-4e54-9608-91f304dc1a94 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:41.887783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120360b3-f1bf-4e54-9608-91f304dc1a94 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:41.895247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 589/1166] 10.222.0.17 () {56 vars in 1115 bytes} [Thu Jun 16 22:49:41 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:43.976995 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b026fb-714c-4044-992b-f019fb00442f None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:43.977543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b026fb-714c-4044-992b-f019fb00442f None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:43.977543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b026fb-714c-4044-992b-f019fb00442f None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:43.979106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 578/1167] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:43.984802 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31db58e-6f29-4208-9313-a3d4584e6d21 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:43.985216 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31db58e-6f29-4208-9313-a3d4584e6d21 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:43.985376 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31db58e-6f29-4208-9313-a3d4584e6d21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:44.002461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a31db58e-6f29-4208-9313-a3d4584e6d21 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:44.003710 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a31db58e-6f29-4208-9313-a3d4584e6d21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:44.070091 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 590/1168] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:44.096000 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b026fb-714c-4044-992b-f019fb00442f None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:44.195738 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b026fb-714c-4044-992b-f019fb00442f None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:44.227705 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b7cc3-5751-4a4c-9eb3-22ab952dc7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:44.228300 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b7cc3-5751-4a4c-9eb3-22ab952dc7da None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:44.228588 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b7cc3-5751-4a4c-9eb3-22ab952dc7da None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:44.228868 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b7cc3-5751-4a4c-9eb3-22ab952dc7da None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:44.229597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b7cc3-5751-4a4c-9eb3-22ab952dc7da None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:44.230977 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b7cc3-5751-4a4c-9eb3-22ab952dc7da None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:44.245669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 579/1169] 10.222.0.17 () {60 vars in 1138 bytes} [Thu Jun 16 22:49:44 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:49:46.376015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d57e42-3055-428c-9939-0afaa1a37310 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:46.376580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d57e42-3055-428c-9939-0afaa1a37310 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:46.376580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d57e42-3055-428c-9939-0afaa1a37310 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:46.377960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 591/1170] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:46.382993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ff9bf-0f43-4097-9e21-aa4d8492a720 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:46.383430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ff9bf-0f43-4097-9e21-aa4d8492a720 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:46.383558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ff9bf-0f43-4097-9e21-aa4d8492a720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:46.400192 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-438ff9bf-0f43-4097-9e21-aa4d8492a720 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:46.401368 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-438ff9bf-0f43-4097-9e21-aa4d8492a720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:46.426440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:46.467293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 580/1171] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:46.490713 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d57e42-3055-428c-9939-0afaa1a37310 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:46.584858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:46.589883 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d57e42-3055-428c-9939-0afaa1a37310 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:46.623197 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de05120-684f-4399-8d6c-223e8c17685a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:46.624055 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de05120-684f-4399-8d6c-223e8c17685a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:46.624338 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de05120-684f-4399-8d6c-223e8c17685a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:46.624617 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de05120-684f-4399-8d6c-223e8c17685a None admin] PATH_INFO: `/v3/domains/41fa89025caf44f8baa6d27486123048` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:46.634696 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de05120-684f-4399-8d6c-223e8c17685a None admin] RBAC: Authorizing `identity:get_domain(domain_id=41fa89025caf44f8baa6d27486123048)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:46.636126 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de05120-684f-4399-8d6c-223e8c17685a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:46.637762 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 592/1172] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jun 16 22:49:46 2022] GET /identity/v3/domains/41fa89025caf44f8baa6d27486123048 => generated 267 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:46.643563 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ff9bf-0f43-4097-9e21-aa4d8492a720 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:46.738277 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ff9bf-0f43-4097-9e21-aa4d8492a720 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:46.770211 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eac2b1-5b42-44a3-b769-3c7c16b0d81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:46.770835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eac2b1-5b42-44a3-b769-3c7c16b0d81e None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:46.771126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eac2b1-5b42-44a3-b769-3c7c16b0d81e None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:46.771409 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eac2b1-5b42-44a3-b769-3c7c16b0d81e None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:46.772388 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eac2b1-5b42-44a3-b769-3c7c16b0d81e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:46.774060 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eac2b1-5b42-44a3-b769-3c7c16b0d81e None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:46.799380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 581/1173] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 22:49:46 2022] POST /identity/v3/projects => generated 361 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 22:49:48.920221 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05f537-06a5-4c3d-bcb5-f5e8491ec7ba None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:48.920767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05f537-06a5-4c3d-bcb5-f5e8491ec7ba None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:48.920866 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05f537-06a5-4c3d-bcb5-f5e8491ec7ba None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:48.922191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 593/1174] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:48.927260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300ae1c-b8bb-4269-8c51-1b0ba28d2f43 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:48.927664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300ae1c-b8bb-4269-8c51-1b0ba28d2f43 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:48.927822 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300ae1c-b8bb-4269-8c51-1b0ba28d2f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:48.944635 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-f300ae1c-b8bb-4269-8c51-1b0ba28d2f43 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:48.945805 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f300ae1c-b8bb-4269-8c51-1b0ba28d2f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:48.955189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:49.011561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 582/1175] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:49.036629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05f537-06a5-4c3d-bcb5-f5e8491ec7ba None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:49.134900 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05f537-06a5-4c3d-bcb5-f5e8491ec7ba None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:49.166928 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b220b7b0-e2d8-4915-8e23-e2ab34058eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:49.167781 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220b7b0-e2d8-4915-8e23-e2ab34058eef None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:49.168069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220b7b0-e2d8-4915-8e23-e2ab34058eef None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:49.168348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b220b7b0-e2d8-4915-8e23-e2ab34058eef None admin] PATH_INFO: `/v3/domains/41fa89025caf44f8baa6d27486123048` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:49.170048 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220b7b0-e2d8-4915-8e23-e2ab34058eef None admin] RBAC: Authorizing `identity:get_domain(domain_id=41fa89025caf44f8baa6d27486123048)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:49.171554 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b220b7b0-e2d8-4915-8e23-e2ab34058eef None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:49.173120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 594/1176] 10.222.0.17 () {56 vars in 1183 bytes} [Thu Jun 16 22:49:49 2022] GET /identity/v3/domains/41fa89025caf44f8baa6d27486123048 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:49.178767 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300ae1c-b8bb-4269-8c51-1b0ba28d2f43 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:49.274837 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300ae1c-b8bb-4269-8c51-1b0ba28d2f43 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:49.307142 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd341f1d-c9e5-4572-ac4e-288e46291ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:49.307800 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd341f1d-c9e5-4572-ac4e-288e46291ece None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:49.308088 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd341f1d-c9e5-4572-ac4e-288e46291ece None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:49.308368 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd341f1d-c9e5-4572-ac4e-288e46291ece None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:49.309077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd341f1d-c9e5-4572-ac4e-288e46291ece None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:49.311109 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd341f1d-c9e5-4572-ac4e-288e46291ece None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:49.326244 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:49.331814 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 583/1177] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 22:49:49 2022] POST /identity/v3/users => generated 311 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:49:51.454555 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cee9c9-9f39-474b-ae8f-78e4fad1bfb7 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:51.455113 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cee9c9-9f39-474b-ae8f-78e4fad1bfb7 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:51.455113 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cee9c9-9f39-474b-ae8f-78e4fad1bfb7 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:51.456508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 595/1178] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:51.461625 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d907988-9ee9-4b5a-834e-69edc8980e77 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:51.462017 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d907988-9ee9-4b5a-834e-69edc8980e77 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:51.462188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d907988-9ee9-4b5a-834e-69edc8980e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:51.478986 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-0d907988-9ee9-4b5a-834e-69edc8980e77 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:51.480151 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-0d907988-9ee9-4b5a-834e-69edc8980e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:51.544139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 584/1179] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:51.566704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cee9c9-9f39-474b-ae8f-78e4fad1bfb7 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:51.595096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:51.666864 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cee9c9-9f39-474b-ae8f-78e4fad1bfb7 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:51.698242 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458121f-31f7-4008-9204-bf5f8851d2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:51.699194 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458121f-31f7-4008-9204-bf5f8851d2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:51.699481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458121f-31f7-4008-9204-bf5f8851d2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:51.700275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b458121f-31f7-4008-9204-bf5f8851d2e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:51.710383 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b458121f-31f7-4008-9204-bf5f8851d2e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:49:51.718496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-b458121f-31f7-4008-9204-bf5f8851d2e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:49:51.718496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:51.718496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:51.718496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:51.718496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:51.718496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:51.718496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:51.718496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:51.718496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:51.718496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:51.718496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:51.718496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:51.718496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:49:51.718496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:49:51.718496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:49:51.718496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:49:51.719675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:49:51.719675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:49:51.719675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:49:51.719675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:49:51.719675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:51.719675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:51.719675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:51.719675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:51.719675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:51.719675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:51.719675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:51.719675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:51.719675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:51.719675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:51.719675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:51.719675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:51.720836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:51.720836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:51.720836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:51.720836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:51.720836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:49:51.720836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:49:51.720836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:49:51.720836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:49:51.720836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:49:51.720836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:49:51.720836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:49:51.720836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:51.720836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 596/1180] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:49:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:49:51.725529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d907988-9ee9-4b5a-834e-69edc8980e77 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:51.820300 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d907988-9ee9-4b5a-834e-69edc8980e77 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:51.847824 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:51.853966 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541e5c4-b006-4a42-9e2b-15335a4ec52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:51.854653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541e5c4-b006-4a42-9e2b-15335a4ec52e None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:51.854944 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541e5c4-b006-4a42-9e2b-15335a4ec52e None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:51.855224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0541e5c4-b006-4a42-9e2b-15335a4ec52e None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:51.855964 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541e5c4-b006-4a42-9e2b-15335a4ec52e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:51.856274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541e5c4-b006-4a42-9e2b-15335a4ec52e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:51.857640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0541e5c4-b006-4a42-9e2b-15335a4ec52e None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:51.865577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 585/1181] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:49:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:51.871237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458121f-31f7-4008-9204-bf5f8851d2e8 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:51.953352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:51.968322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b458121f-31f7-4008-9204-bf5f8851d2e8 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:52.001310 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa790c2-cd0d-4706-8dc6-83ea608b9676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:52.002221 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa790c2-cd0d-4706-8dc6-83ea608b9676 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:52.002510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa790c2-cd0d-4706-8dc6-83ea608b9676 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:52.002842 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa790c2-cd0d-4706-8dc6-83ea608b9676 None admin] PATH_INFO: `/v3/users/736edda8b6f84772bffc9f4ee83c187c` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:52.015928 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa790c2-cd0d-4706-8dc6-83ea608b9676 None admin] RBAC: Authorizing `identity:get_user(user_id=736edda8b6f84772bffc9f4ee83c187c)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:52.017349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa790c2-cd0d-4706-8dc6-83ea608b9676 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:52.019017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 597/1182] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:49:51 2022] GET /identity/v3/users/736edda8b6f84772bffc9f4ee83c187c => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:52.024880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541e5c4-b006-4a42-9e2b-15335a4ec52e None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:52.119917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0541e5c4-b006-4a42-9e2b-15335a4ec52e None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:52.152130 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9e703-6bef-4599-86e6-dd82b9cffaa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:52.152785 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9e703-6bef-4599-86e6-dd82b9cffaa8 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:52.153075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9e703-6bef-4599-86e6-dd82b9cffaa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:52.153354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e9e703-6bef-4599-86e6-dd82b9cffaa8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:52.209203 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:52.220038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9e703-6bef-4599-86e6-dd82b9cffaa8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:52.221477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8e9e703-6bef-4599-86e6-dd82b9cffaa8 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:52.316716 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 586/1183] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:49:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:49:52.322571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa790c2-cd0d-4706-8dc6-83ea608b9676 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:52.419293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa790c2-cd0d-4706-8dc6-83ea608b9676 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:52.453361 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b4dba5-70d3-4a82-b492-f3ee9539e8a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:52.454218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4dba5-70d3-4a82-b492-f3ee9539e8a6 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:52.454502 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4dba5-70d3-4a82-b492-f3ee9539e8a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:52.454970 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b4dba5-70d3-4a82-b492-f3ee9539e8a6 None admin] PATH_INFO: `/v3/projects/eecfe1f7da87497fad44c37396104915` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:52.456143 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4dba5-70d3-4a82-b492-f3ee9539e8a6 None admin] RBAC: Authorizing `identity:get_project(project_id=eecfe1f7da87497fad44c37396104915)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:52.457561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b4dba5-70d3-4a82-b492-f3ee9539e8a6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:52.459282 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 598/1184] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:49:52 2022] GET /identity/v3/projects/eecfe1f7da87497fad44c37396104915 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:52.465360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9e703-6bef-4599-86e6-dd82b9cffaa8 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:52.553126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:52.561857 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e9e703-6bef-4599-86e6-dd82b9cffaa8 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:52.594163 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9965425-fc1d-40b9-bbc5-23a160ede00b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:52.594822 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9965425-fc1d-40b9-bbc5-23a160ede00b None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:52.595105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9965425-fc1d-40b9-bbc5-23a160ede00b None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:52.595379 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9965425-fc1d-40b9-bbc5-23a160ede00b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:52.596192 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9965425-fc1d-40b9-bbc5-23a160ede00b None admin] RBAC: Adding query filter params (role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.project.id=eecfe1f7da87497fad44c37396104915, user.id=736edda8b6f84772bffc9f4ee83c187c) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:52.596505 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9965425-fc1d-40b9-bbc5-23a160ede00b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:52.597920 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9965425-fc1d-40b9-bbc5-23a160ede00b None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:52.603135 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 587/1185] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:49:52 2022] GET /identity/v3/role_assignments?user.id=736edda8b6f84772bffc9f4ee83c187c&scope.project.id=eecfe1f7da87497fad44c37396104915&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:54.681231 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfba398-5e9a-4842-9abb-ca79778344a2 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:54.681778 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfba398-5e9a-4842-9abb-ca79778344a2 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:54.681778 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfba398-5e9a-4842-9abb-ca79778344a2 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:54.683185 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 599/1186] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:54.688579 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36ebfd-f80b-44f3-b58b-054c0dcc738f None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:54.688962 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36ebfd-f80b-44f3-b58b-054c0dcc738f None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:54.689160 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36ebfd-f80b-44f3-b58b-054c0dcc738f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:54.706094 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-fc36ebfd-f80b-44f3-b58b-054c0dcc738f None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:54.707308 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-fc36ebfd-f80b-44f3-b58b-054c0dcc738f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:54.772718 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 588/1187] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:54.795023 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfba398-5e9a-4842-9abb-ca79778344a2 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:54.809832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:54.893854 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfba398-5e9a-4842-9abb-ca79778344a2 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:54.926068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffc8db-c496-4331-ab04-28a29f02508d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:54.926877 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffc8db-c496-4331-ab04-28a29f02508d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:54.927163 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffc8db-c496-4331-ab04-28a29f02508d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:54.927443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ffc8db-c496-4331-ab04-28a29f02508d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:54.937765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ffc8db-c496-4331-ab04-28a29f02508d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:49:54.945276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-24ffc8db-c496-4331-ab04-28a29f02508d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:49:54.945276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:54.945276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:54.945276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:54.945276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:54.945276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:54.945276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:54.945276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:54.945276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:54.945276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:54.945276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:54.945276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:54.945276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:49:54.945276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:49:54.945276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:49:54.945276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:49:54.946363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:49:54.946363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:49:54.946363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:49:54.946363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:49:54.946363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:54.946363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:54.946363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:54.946363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:54.946363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:54.946363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:54.946363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:54.946363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:54.946363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:54.946363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:54.946363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:54.946363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:54.947470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:54.947470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:54.947470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:54.947470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:54.947470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:49:54.947470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:49:54.947470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:49:54.947470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:49:54.947470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:49:54.947470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:49:54.947470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:49:54.947470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:49:54.947470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 600/1188] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:49:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:49:54.953300 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36ebfd-f80b-44f3-b58b-054c0dcc738f None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:55.035859 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:55.050411 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36ebfd-f80b-44f3-b58b-054c0dcc738f None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:55.083719 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0b16d-2816-4cbd-a1ef-aa4fabc18936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:55.084374 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0b16d-2816-4cbd-a1ef-aa4fabc18936 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:55.084658 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0b16d-2816-4cbd-a1ef-aa4fabc18936 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:55.084935 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e0b16d-2816-4cbd-a1ef-aa4fabc18936 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:55.085671 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e0b16d-2816-4cbd-a1ef-aa4fabc18936 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:55.085979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e0b16d-2816-4cbd-a1ef-aa4fabc18936 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:55.087351 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e0b16d-2816-4cbd-a1ef-aa4fabc18936 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:55.095247 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 589/1189] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:49:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:55.100851 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffc8db-c496-4331-ab04-28a29f02508d None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:55.169449 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:55.198479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ffc8db-c496-4331-ab04-28a29f02508d None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:55.231563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853f5a6-ed34-44e9-8531-2611f316308b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:55.232439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853f5a6-ed34-44e9-8531-2611f316308b None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:55.232723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853f5a6-ed34-44e9-8531-2611f316308b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:55.233001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8853f5a6-ed34-44e9-8531-2611f316308b None admin] PATH_INFO: `/v3/users/736edda8b6f84772bffc9f4ee83c187c` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:55.234182 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853f5a6-ed34-44e9-8531-2611f316308b None admin] RBAC: Authorizing `identity:get_user(user_id=736edda8b6f84772bffc9f4ee83c187c)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:55.235646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8853f5a6-ed34-44e9-8531-2611f316308b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:55.237287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 601/1190] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:49:55 2022] GET /identity/v3/users/736edda8b6f84772bffc9f4ee83c187c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:55.242995 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0b16d-2816-4cbd-a1ef-aa4fabc18936 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:55.338363 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e0b16d-2816-4cbd-a1ef-aa4fabc18936 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:55.370220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac5a69c-39e1-4e4a-81e7-384cf3f71c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:55.371104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5a69c-39e1-4e4a-81e7-384cf3f71c2f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:55.371928 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5a69c-39e1-4e4a-81e7-384cf3f71c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:55.372208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac5a69c-39e1-4e4a-81e7-384cf3f71c2f None admin] PATH_INFO: `/v3/projects/eecfe1f7da87497fad44c37396104915` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:55.373371 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5a69c-39e1-4e4a-81e7-384cf3f71c2f None admin] RBAC: Authorizing `identity:get_project(project_id=eecfe1f7da87497fad44c37396104915)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:55.374810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac5a69c-39e1-4e4a-81e7-384cf3f71c2f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:55.376486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 590/1191] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:49:55 2022] GET /identity/v3/projects/eecfe1f7da87497fad44c37396104915 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:55.382511 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853f5a6-ed34-44e9-8531-2611f316308b None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:55.481090 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8853f5a6-ed34-44e9-8531-2611f316308b None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:55.513614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfed54f1-62e0-4fb7-b022-a6caf1cd11c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:55.514481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed54f1-62e0-4fb7-b022-a6caf1cd11c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:55.514789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed54f1-62e0-4fb7-b022-a6caf1cd11c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:55.515061 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfed54f1-62e0-4fb7-b022-a6caf1cd11c5 None admin] PATH_INFO: `/v3/projects/eecfe1f7da87497fad44c37396104915/users/736edda8b6f84772bffc9f4ee83c187c/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:55.516972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed54f1-62e0-4fb7-b022-a6caf1cd11c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=eecfe1f7da87497fad44c37396104915, user_id=736edda8b6f84772bffc9f4ee83c187c, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:55.518375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfed54f1-62e0-4fb7-b022-a6caf1cd11c5 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:55.526917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:55.532438 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 602/1192] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:49:55 2022] PUT /identity/v3/projects/eecfe1f7da87497fad44c37396104915/users/736edda8b6f84772bffc9f4ee83c187c/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:49:57.634167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58a521-64f6-4b7c-9dbf-159510ddec68 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:57.634708 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58a521-64f6-4b7c-9dbf-159510ddec68 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:57.634817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd58a521-64f6-4b7c-9dbf-159510ddec68 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:57.636123 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 591/1193] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:49:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:49:57.641692 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fce98-2666-406e-9879-8e6fb7010206 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:57.642081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fce98-2666-406e-9879-8e6fb7010206 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:57.642256 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873fce98-2666-406e-9879-8e6fb7010206 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:57.659423 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-873fce98-2666-406e-9879-8e6fb7010206 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:49:57.660611 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-873fce98-2666-406e-9879-8e6fb7010206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:49:57.725096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 603/1194] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:49:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:49:57.748590 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58a521-64f6-4b7c-9dbf-159510ddec68 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:57.767255 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:57.847262 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd58a521-64f6-4b7c-9dbf-159510ddec68 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:57.879178 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721cf3d-8aa3-44ba-96b3-63f9e2a7399f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:57.879959 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721cf3d-8aa3-44ba-96b3-63f9e2a7399f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:57.880246 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721cf3d-8aa3-44ba-96b3-63f9e2a7399f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:57.880522 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721cf3d-8aa3-44ba-96b3-63f9e2a7399f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:57.890843 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5721cf3d-8aa3-44ba-96b3-63f9e2a7399f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:49:57.898433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-5721cf3d-8aa3-44ba-96b3-63f9e2a7399f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:49:57.898433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:49:57.898433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:49:57.898433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:49:57.898433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:49:57.898433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:49:57.898433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:49:57.898433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:49:57.898433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:49:57.898433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:49:57.898433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:49:57.898433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:49:57.898433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:49:57.898433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:49:57.898433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:49:57.898433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:49:57.899498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:49:57.899498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:49:57.899498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:49:57.899498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:49:57.899498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:49:57.899498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:49:57.899498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:49:57.899498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:49:57.899498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:49:57.899498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:49:57.899498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:49:57.899498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:49:57.899498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:49:57.899498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:49:57.899498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:49:57.899498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:49:57.900515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:49:57.900515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:49:57.900515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:49:57.900515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:49:57.900515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:49:57.900515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:49:57.900515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:49:57.900515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:49:57.900515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:49:57.900515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:49:57.900515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:49:57.900515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:49:57.900515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 592/1195] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:49:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:49:57.905991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fce98-2666-406e-9879-8e6fb7010206 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:58.001393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873fce98-2666-406e-9879-8e6fb7010206 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:58.029110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:58.035080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bec58f-b432-49b7-acd4-2673e131d5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:58.035738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bec58f-b432-49b7-acd4-2673e131d5b8 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:58.036026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bec58f-b432-49b7-acd4-2673e131d5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:58.036307 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bec58f-b432-49b7-acd4-2673e131d5b8 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:58.037042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bec58f-b432-49b7-acd4-2673e131d5b8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:58.037353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bec58f-b432-49b7-acd4-2673e131d5b8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:58.038727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bec58f-b432-49b7-acd4-2673e131d5b8 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:58.047086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 604/1196] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:49:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:58.052777 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721cf3d-8aa3-44ba-96b3-63f9e2a7399f None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:58.126249 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:58.149912 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721cf3d-8aa3-44ba-96b3-63f9e2a7399f None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:58.182301 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38876c7d-3def-454d-b11b-1580c3b7d9ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:58.183192 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38876c7d-3def-454d-b11b-1580c3b7d9ff None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:58.183520 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38876c7d-3def-454d-b11b-1580c3b7d9ff None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:58.184283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38876c7d-3def-454d-b11b-1580c3b7d9ff None admin] PATH_INFO: `/v3/users/736edda8b6f84772bffc9f4ee83c187c` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:58.185457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38876c7d-3def-454d-b11b-1580c3b7d9ff None admin] RBAC: Authorizing `identity:get_user(user_id=736edda8b6f84772bffc9f4ee83c187c)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:58.186899 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38876c7d-3def-454d-b11b-1580c3b7d9ff None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:58.188530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 593/1197] 10.222.0.17 () {56 vars in 1177 bytes} [Thu Jun 16 22:49:58 2022] GET /identity/v3/users/736edda8b6f84772bffc9f4ee83c187c => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:58.194172 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bec58f-b432-49b7-acd4-2673e131d5b8 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:58.290967 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bec58f-b432-49b7-acd4-2673e131d5b8 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:58.324017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce567e-f11c-4d0f-854c-0d17f6ccb397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:58.324659 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce567e-f11c-4d0f-854c-0d17f6ccb397 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:58.324952 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce567e-f11c-4d0f-854c-0d17f6ccb397 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:58.325235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ce567e-f11c-4d0f-854c-0d17f6ccb397 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:58.381166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:58.390718 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce567e-f11c-4d0f-854c-0d17f6ccb397 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:58.392157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ce567e-f11c-4d0f-854c-0d17f6ccb397 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:58.486733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 605/1198] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:49:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:49:58.492496 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38876c7d-3def-454d-b11b-1580c3b7d9ff None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:58.589107 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38876c7d-3def-454d-b11b-1580c3b7d9ff None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:58.622513 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7072c36-61be-4464-993a-6580ee2330d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:58.623391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7072c36-61be-4464-993a-6580ee2330d3 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:58.623678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7072c36-61be-4464-993a-6580ee2330d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:58.623958 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7072c36-61be-4464-993a-6580ee2330d3 None admin] PATH_INFO: `/v3/projects/eecfe1f7da87497fad44c37396104915` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:58.625125 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7072c36-61be-4464-993a-6580ee2330d3 None admin] RBAC: Authorizing `identity:get_project(project_id=eecfe1f7da87497fad44c37396104915)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:58.626554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7072c36-61be-4464-993a-6580ee2330d3 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:58.628270 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 594/1199] 10.222.0.17 () {56 vars in 1186 bytes} [Thu Jun 16 22:49:58 2022] GET /identity/v3/projects/eecfe1f7da87497fad44c37396104915 => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:49:58.634276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce567e-f11c-4d0f-854c-0d17f6ccb397 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:49:58.721586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:49:58.730864 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ce567e-f11c-4d0f-854c-0d17f6ccb397 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:49:58.763731 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53df5089-0d75-48c7-95dd-8aaa39030445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:49:58.764319 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df5089-0d75-48c7-95dd-8aaa39030445 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:49:58.764604 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df5089-0d75-48c7-95dd-8aaa39030445 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:49:58.764884 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53df5089-0d75-48c7-95dd-8aaa39030445 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:49:58.765701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df5089-0d75-48c7-95dd-8aaa39030445 None admin] RBAC: Adding query filter params (role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.project.id=eecfe1f7da87497fad44c37396104915, user.id=736edda8b6f84772bffc9f4ee83c187c) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:49:58.766016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df5089-0d75-48c7-95dd-8aaa39030445 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:49:58.767454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53df5089-0d75-48c7-95dd-8aaa39030445 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:49:58.773321 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 606/1200] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:49:58 2022] GET /identity/v3/role_assignments?user.id=736edda8b6f84772bffc9f4ee83c187c&scope.project.id=eecfe1f7da87497fad44c37396104915&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 608 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:50:08.618381 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=94 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:50:08.770465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=22 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:50:54.955425 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c89092-67ca-4ac3-9c0d-1da4762e2b81 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:50:54.956118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c89092-67ca-4ac3-9c0d-1da4762e2b81 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:50:54.956118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c89092-67ca-4ac3-9c0d-1da4762e2b81 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:50:54.957239 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 595/1201] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:50:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:50:54.965325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509be4ef-3828-47ef-a603-4e69ef0958be None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:50:54.965325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509be4ef-3828-47ef-a603-4e69ef0958be None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:50:54.965325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509be4ef-3828-47ef-a603-4e69ef0958be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:50:54.986699 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-509be4ef-3828-47ef-a603-4e69ef0958be None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:50:54.987920 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-509be4ef-3828-47ef-a603-4e69ef0958be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:50:55.075549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 607/1202] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:50:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 22:50:55.106319 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c89092-67ca-4ac3-9c0d-1da4762e2b81 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:50:55.235106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c89092-67ca-4ac3-9c0d-1da4762e2b81 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:50:55.272113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82edab-8f44-43a4-a565-6525449e82c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:50:55.273182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82edab-8f44-43a4-a565-6525449e82c7 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:50:55.273447 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82edab-8f44-43a4-a565-6525449e82c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:50:55.273756 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba82edab-8f44-43a4-a565-6525449e82c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:50:55.284538 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82edab-8f44-43a4-a565-6525449e82c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:50:55.286161 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba82edab-8f44-43a4-a565-6525449e82c7 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:50:55.307144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-ba82edab-8f44-43a4-a565-6525449e82c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:50:55.307144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:50:55.307144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:50:55.307144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:50:55.307144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:50:55.307144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:50:55.307144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:50:55.307144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:50:55.307144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:50:55.307144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:50:55.307144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:50:55.307144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:50:55.307144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:50:55.307144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:50:55.307144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:50:55.309051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:50:55.309051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:50:55.309051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:50:55.309051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:50:55.309051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:50:55.309051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:50:55.309051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:50:55.309051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:50:55.309051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:50:55.309051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:50:55.309051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:50:55.309051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:50:55.309051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:50:55.309051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:50:55.309051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:50:55.309051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:50:55.309051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:50:55.310158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:50:55.310158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:50:55.310158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:50:55.310158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:50:55.310158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 596/1203] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:50:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:50:55.315067 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509be4ef-3828-47ef-a603-4e69ef0958be None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:50:55.464890 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509be4ef-3828-47ef-a603-4e69ef0958be None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:50:55.501701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:50:55.509501 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d434d60-a84e-489e-a8df-294513098f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:50:55.510258 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d434d60-a84e-489e-a8df-294513098f16 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:50:55.510680 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d434d60-a84e-489e-a8df-294513098f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:50:55.511029 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d434d60-a84e-489e-a8df-294513098f16 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:50:55.511823 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d434d60-a84e-489e-a8df-294513098f16 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:50:55.512322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d434d60-a84e-489e-a8df-294513098f16 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:50:55.513673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d434d60-a84e-489e-a8df-294513098f16 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:50:55.528535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 608/1204] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:50:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:50:55.534317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82edab-8f44-43a4-a565-6525449e82c7 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:50:55.599641 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:50:55.673916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba82edab-8f44-43a4-a565-6525449e82c7 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:50:55.713386 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf417dd-a9ea-4ae3-99e1-05ae59775c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:50:55.714257 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf417dd-a9ea-4ae3-99e1-05ae59775c39 None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:50:55.714512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf417dd-a9ea-4ae3-99e1-05ae59775c39 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:50:55.714812 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf417dd-a9ea-4ae3-99e1-05ae59775c39 None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:50:55.715685 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf417dd-a9ea-4ae3-99e1-05ae59775c39 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:50:55.717225 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf417dd-a9ea-4ae3-99e1-05ae59775c39 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:50:55.752324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 597/1205] 10.222.0.17 () {60 vars in 1132 bytes} [Thu Jun 16 22:50:55 2022] POST /identity/v3/users => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 22:50:58.031618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3fc5e-54e3-469c-8284-9856fdb5c64f None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:50:58.032192 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3fc5e-54e3-469c-8284-9856fdb5c64f None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:50:58.032192 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d3fc5e-54e3-469c-8284-9856fdb5c64f None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:50:58.035682 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 609/1206] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:50:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:50:58.038823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763d939-1b2d-46a7-9ad3-971e6cd31a19 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:50:58.039795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763d939-1b2d-46a7-9ad3-971e6cd31a19 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:50:58.039795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3763d939-1b2d-46a7-9ad3-971e6cd31a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:50:58.058960 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-3763d939-1b2d-46a7-9ad3-971e6cd31a19 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:50:58.060288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-3763d939-1b2d-46a7-9ad3-971e6cd31a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:50:58.150349 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 598/1207] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:50:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:50:58.174890 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3fc5e-54e3-469c-8284-9856fdb5c64f None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:50:58.314975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d3fc5e-54e3-469c-8284-9856fdb5c64f None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:50:58.357933 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6aff6c-49a5-4acb-b7e3-1c456ddd1729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:50:58.358888 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6aff6c-49a5-4acb-b7e3-1c456ddd1729 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:50:58.359222 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6aff6c-49a5-4acb-b7e3-1c456ddd1729 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:50:58.359509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6aff6c-49a5-4acb-b7e3-1c456ddd1729 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:50:58.373622 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6aff6c-49a5-4acb-b7e3-1c456ddd1729 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:50:58.382402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-ed6aff6c-49a5-4acb-b7e3-1c456ddd1729 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:50:58.382402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:50:58.382402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:50:58.382402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:50:58.382402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:50:58.382402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:50:58.382402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:50:58.382402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:50:58.382402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:50:58.382402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:50:58.382402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:50:58.382402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:50:58.382402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:50:58.382402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:50:58.382402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:50:58.382402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:50:58.383454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:50:58.383454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:50:58.383454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:50:58.383454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:50:58.383454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:50:58.383454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:50:58.383454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:50:58.383454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:50:58.383454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:50:58.383454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:50:58.383454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:50:58.383454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:50:58.383454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:50:58.383454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:50:58.383454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:50:58.383454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:50:58.384286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:50:58.384286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:50:58.384286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:50:58.384286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:50:58.384286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:50:58.384286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:50:58.384286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:50:58.384286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:50:58.384286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:50:58.384286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:50:58.384286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:50:58.384286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:50:58.384286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 610/1208] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:50:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:50:58.389839 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763d939-1b2d-46a7-9ad3-971e6cd31a19 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:50:58.446954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:50:58.518306 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3763d939-1b2d-46a7-9ad3-971e6cd31a19 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:50:58.558102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf94264-b409-4462-a807-d536905ba562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:50:58.558809 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf94264-b409-4462-a807-d536905ba562 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:50:58.559096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf94264-b409-4462-a807-d536905ba562 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:50:58.559380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf94264-b409-4462-a807-d536905ba562 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:50:58.560123 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf94264-b409-4462-a807-d536905ba562 None admin] RBAC: Adding query filter params (name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:50:58.560443 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf94264-b409-4462-a807-d536905ba562 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:50:58.561802 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf94264-b409-4462-a807-d536905ba562 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:50:58.570794 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 599/1209] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:50:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:50:58.576252 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6aff6c-49a5-4acb-b7e3-1c456ddd1729 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:50:58.632612 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:50:58.692453 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6aff6c-49a5-4acb-b7e3-1c456ddd1729 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:50:58.743741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879a9e6-6cf5-4ac4-9847-504634cb50ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:50:58.744629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879a9e6-6cf5-4ac4-9847-504634cb50ae None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:50:58.744936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879a9e6-6cf5-4ac4-9847-504634cb50ae None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:50:58.745240 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2879a9e6-6cf5-4ac4-9847-504634cb50ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:50:58.758366 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879a9e6-6cf5-4ac4-9847-504634cb50ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:50:58.759964 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2879a9e6-6cf5-4ac4-9847-504634cb50ae None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:50:58.779777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-2879a9e6-6cf5-4ac4-9847-504634cb50ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:50:58.779777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:50:58.779777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:50:58.779777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:50:58.779777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:50:58.779777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:50:58.779777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:50:58.779777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:50:58.779777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:50:58.779777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:50:58.779777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:50:58.779777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:50:58.779777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:50:58.779777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:50:58.779777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:50:58.780856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:50:58.780856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:50:58.780856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:50:58.780856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:50:58.780856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:50:58.780856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:50:58.780856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:50:58.780856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:50:58.780856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:50:58.780856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:50:58.780856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:50:58.780856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:50:58.780856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:50:58.780856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:50:58.780856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:50:58.780856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:50:58.780856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:50:58.786503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:50:58.786503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:50:58.786503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:50:58.786503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:50:58.786503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 611/1210] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:50:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:50:58.787057 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf94264-b409-4462-a807-d536905ba562 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:50:58.909288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:50:58.913245 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf94264-b409-4462-a807-d536905ba562 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:50:58.951137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e9e28-ab29-4c96-93ac-8856f351d182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:50:58.953568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e9e28-ab29-4c96-93ac-8856f351d182 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:50:58.953568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e9e28-ab29-4c96-93ac-8856f351d182 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:50:58.953568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846e9e28-ab29-4c96-93ac-8856f351d182 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:50:58.953568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e9e28-ab29-4c96-93ac-8856f351d182 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:50:58.953568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e9e28-ab29-4c96-93ac-8856f351d182 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:50:58.955136 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846e9e28-ab29-4c96-93ac-8856f351d182 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:50:58.971101 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 600/1211] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:50:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:50:58.978239 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879a9e6-6cf5-4ac4-9847-504634cb50ae None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:50:59.096712 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2879a9e6-6cf5-4ac4-9847-504634cb50ae None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:50:59.098500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:50:59.138205 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8520059-3831-46bb-972a-34127e0ffcd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:50:59.139161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8520059-3831-46bb-972a-34127e0ffcd3 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:50:59.139557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8520059-3831-46bb-972a-34127e0ffcd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:50:59.139889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8520059-3831-46bb-972a-34127e0ffcd3 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:50:59.147645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8520059-3831-46bb-972a-34127e0ffcd3 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:50:59.149388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8520059-3831-46bb-972a-34127e0ffcd3 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:50:59.165614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-f8520059-3831-46bb-972a-34127e0ffcd3 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 22:50:59.165614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:50:59.165614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:50:59.165614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:50:59.165614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:50:59.165614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:50:59.165614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:50:59.165614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:50:59.165614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:50:59.165614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:50:59.165614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:50:59.165614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:50:59.165614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:50:59.165614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:50:59.165614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:50:59.165614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:50:59.166886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:50:59.166886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:50:59.166886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:50:59.166886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:50:59.166886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:50:59.166886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:50:59.166886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:50:59.166886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:50:59.166886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:50:59.166886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:50:59.166886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:50:59.166886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:50:59.166886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:50:59.166886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:50:59.166886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:50:59.166886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:50:59.166886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:50:59.166886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:50:59.168092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:50:59.168092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:50:59.168092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:50:59.168092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:50:59.168092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:50:59.168092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:50:59.168092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:50:59.168092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:50:59.168092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 22:50:59.168092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:50:59.174031 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 612/1212] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:50:58 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:50:59.175595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e9e28-ab29-4c96-93ac-8856f351d182 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:50:59.297485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846e9e28-ab29-4c96-93ac-8856f351d182 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:50:59.335638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4284578-2ab4-4991-a0ce-f81620a5a89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:50:59.336334 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4284578-2ab4-4991-a0ce-f81620a5a89e None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:50:59.336619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4284578-2ab4-4991-a0ce-f81620a5a89e None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:50:59.336897 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4284578-2ab4-4991-a0ce-f81620a5a89e None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:50:59.337739 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4284578-2ab4-4991-a0ce-f81620a5a89e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:50:59.338052 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4284578-2ab4-4991-a0ce-f81620a5a89e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:50:59.339561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4284578-2ab4-4991-a0ce-f81620a5a89e None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:50:59.359359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 601/1213] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jun 16 22:50:59 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:50:59.365610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8520059-3831-46bb-972a-34127e0ffcd3 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:50:59.482116 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8520059-3831-46bb-972a-34127e0ffcd3 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:50:59.526569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13749da2-062d-4487-9b79-faf16c1befce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:50:59.527366 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13749da2-062d-4487-9b79-faf16c1befce None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:50:59.527750 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13749da2-062d-4487-9b79-faf16c1befce None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:50:59.528079 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13749da2-062d-4487-9b79-faf16c1befce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:50:59.565890 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:50:59.609868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13749da2-062d-4487-9b79-faf16c1befce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:50:59.611593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13749da2-062d-4487-9b79-faf16c1befce None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:50:59.730263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 613/1214] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:50:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:50:59.737188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4284578-2ab4-4991-a0ce-f81620a5a89e None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:50:59.764329 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:50:59.868031 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4284578-2ab4-4991-a0ce-f81620a5a89e None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:50:59.907939 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79717dd4-8ea1-4f40-a427-f8df939dd17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:50:59.908786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79717dd4-8ea1-4f40-a427-f8df939dd17c None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:50:59.909113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79717dd4-8ea1-4f40-a427-f8df939dd17c None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:50:59.909469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79717dd4-8ea1-4f40-a427-f8df939dd17c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:50:59.923025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79717dd4-8ea1-4f40-a427-f8df939dd17c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:50:59.924661 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79717dd4-8ea1-4f40-a427-f8df939dd17c None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:50:59.943477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-79717dd4-8ea1-4f40-a427-f8df939dd17c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:50:59.943477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:50:59.943477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:50:59.943477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:50:59.943477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:50:59.943477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:50:59.943477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:50:59.943477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:50:59.943477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:50:59.943477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:50:59.943477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:50:59.943477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:50:59.943477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:50:59.943477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:50:59.943477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:50:59.944442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:50:59.944442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:50:59.944442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:50:59.944442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:50:59.944442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:50:59.944442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:50:59.944442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:50:59.944442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:50:59.944442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:50:59.944442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:50:59.944442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:50:59.944442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:50:59.944442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:50:59.944442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:50:59.944442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:50:59.944442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:50:59.944442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:50:59.945353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:50:59.945353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:50:59.945353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:50:59.945353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:50:59.949983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 602/1215] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:50:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:50:59.951265 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13749da2-062d-4487-9b79-faf16c1befce None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:00.026373 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:00.070626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13749da2-062d-4487-9b79-faf16c1befce None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:00.116792 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3da965-ce82-44df-a8f0-bd4217178ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:00.117465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3da965-ce82-44df-a8f0-bd4217178ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:00.117741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3da965-ce82-44df-a8f0-bd4217178ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:00.118023 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3da965-ce82-44df-a8f0-bd4217178ac2 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:00.118778 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3da965-ce82-44df-a8f0-bd4217178ac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:00.119094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3da965-ce82-44df-a8f0-bd4217178ac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:00.121146 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3da965-ce82-44df-a8f0-bd4217178ac2 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:00.140431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 614/1216] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:50:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:00.141713 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79717dd4-8ea1-4f40-a427-f8df939dd17c None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:00.242297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:00.285199 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79717dd4-8ea1-4f40-a427-f8df939dd17c None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:00.326573 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a61223-b059-4a43-8341-0688c8aa1d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:00.327475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a61223-b059-4a43-8341-0688c8aa1d02 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:00.327765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a61223-b059-4a43-8341-0688c8aa1d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:00.328046 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a61223-b059-4a43-8341-0688c8aa1d02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:00.335768 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a61223-b059-4a43-8341-0688c8aa1d02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:00.337398 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a61223-b059-4a43-8341-0688c8aa1d02 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:00.353798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-23a61223-b059-4a43-8341-0688c8aa1d02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:51:00.353798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:00.353798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:00.353798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:00.353798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:00.353798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:00.353798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:00.353798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:00.353798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:00.353798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:00.353798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:00.353798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:00.353798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:51:00.353798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:51:00.353798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:51:00.354888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:51:00.354888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:00.354888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:00.354888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:00.354888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:00.354888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:00.354888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:00.354888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:00.354888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:00.354888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:00.354888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:00.354888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:00.354888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:00.354888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:00.354888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:00.354888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:00.354888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:00.360909 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:51:00.360909 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:51:00.360909 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:51:00.360909 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:51:00.360909 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:51:00.360909 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:51:00.360909 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:51:00.360909 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:51:00.360909 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 603/1217] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:51:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:00.361426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3da965-ce82-44df-a8f0-bd4217178ac2 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:00.489337 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3da965-ce82-44df-a8f0-bd4217178ac2 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:00.516882 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:00.533296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0374559-e33e-41c4-82f3-d46d49291e6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:00.534168 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0374559-e33e-41c4-82f3-d46d49291e6d None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:00.534469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0374559-e33e-41c4-82f3-d46d49291e6d None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:00.534899 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0374559-e33e-41c4-82f3-d46d49291e6d None admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:00.535792 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0374559-e33e-41c4-82f3-d46d49291e6d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:00.536212 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0374559-e33e-41c4-82f3-d46d49291e6d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:00.538005 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0374559-e33e-41c4-82f3-d46d49291e6d None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:00.558004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 615/1218] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:51:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:51:00.559993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a61223-b059-4a43-8341-0688c8aa1d02 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:00.684321 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a61223-b059-4a43-8341-0688c8aa1d02 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:00.716678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:00.725576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8388f2-6ef8-47a6-be43-9ca822b8e19a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:00.726276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8388f2-6ef8-47a6-be43-9ca822b8e19a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:00.726567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8388f2-6ef8-47a6-be43-9ca822b8e19a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:00.726896 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8388f2-6ef8-47a6-be43-9ca822b8e19a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:00.727732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8388f2-6ef8-47a6-be43-9ca822b8e19a None admin] RBAC: Adding query filter params (role.id=ec841a43f9924fa1b2209bc5ecea5e04, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=f9593939641c4c7797c1981206b40e7d) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:00.728049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8388f2-6ef8-47a6-be43-9ca822b8e19a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:00.729531 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8388f2-6ef8-47a6-be43-9ca822b8e19a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:00.736013 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 604/1219] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:51:00 2022] GET /identity/v3/role_assignments?user.id=f9593939641c4c7797c1981206b40e7d&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=ec841a43f9924fa1b2209bc5ecea5e04 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:51:02.891974 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1a3e7-59df-44cb-b95c-ed749d4e620a None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:02.892536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1a3e7-59df-44cb-b95c-ed749d4e620a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:02.892536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1a3e7-59df-44cb-b95c-ed749d4e620a None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:02.894043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 616/1220] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:51:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:51:02.899533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d86bb-5944-4053-80e7-d35d07e6bb4a None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:02.899936 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d86bb-5944-4053-80e7-d35d07e6bb4a None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:02.900102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0d86bb-5944-4053-80e7-d35d07e6bb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:02.916419 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-ee0d86bb-5944-4053-80e7-d35d07e6bb4a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:51:02.918131 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-ee0d86bb-5944-4053-80e7-d35d07e6bb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:51:02.988682 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 605/1221] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:51:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:51:03.011708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1a3e7-59df-44cb-b95c-ed749d4e620a None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:03.124761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1a3e7-59df-44cb-b95c-ed749d4e620a None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:03.168945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853b481-0034-44a2-9d2c-faf79349f07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:03.169676 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853b481-0034-44a2-9d2c-faf79349f07f None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:03.169959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853b481-0034-44a2-9d2c-faf79349f07f None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:03.170234 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7853b481-0034-44a2-9d2c-faf79349f07f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:03.180486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7853b481-0034-44a2-9d2c-faf79349f07f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:51:03.188253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-7853b481-0034-44a2-9d2c-faf79349f07f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:51:03.188253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:03.188253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:03.188253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:03.188253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:03.188253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:03.188253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:03.188253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:03.188253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:03.188253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:03.188253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:03.188253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:03.188253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:51:03.188253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:51:03.188253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:51:03.188253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:51:03.189158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:51:03.189158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:51:03.189158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:51:03.189158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:51:03.189158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:03.189158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:03.189158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:03.189158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:03.189158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:03.189158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:03.189158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:03.189158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:03.189158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:03.189158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:03.189158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:03.189158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:03.190247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:03.190247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:03.190247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:03.190247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:03.190247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:51:03.190247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:51:03.190247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:51:03.190247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:51:03.190247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:51:03.190247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:51:03.190247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:51:03.190247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:03.190247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 617/1222] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:51:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:03.195507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d86bb-5944-4053-80e7-d35d07e6bb4a None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:03.313706 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0d86bb-5944-4053-80e7-d35d07e6bb4a None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:03.338422 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:03.349683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e9964-ceef-4591-ba78-a43961e54cd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:03.350360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e9964-ceef-4591-ba78-a43961e54cd6 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:03.350683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e9964-ceef-4591-ba78-a43961e54cd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:03.350964 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e9964-ceef-4591-ba78-a43961e54cd6 None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:03.351711 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e9964-ceef-4591-ba78-a43961e54cd6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:03.352022 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e9964-ceef-4591-ba78-a43961e54cd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:03.353384 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5e9964-ceef-4591-ba78-a43961e54cd6 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:03.361999 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 606/1223] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:51:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:03.367769 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853b481-0034-44a2-9d2c-faf79349f07f None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:03.412458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:03.474066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7853b481-0034-44a2-9d2c-faf79349f07f None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:03.518180 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b0cf3-9b59-4980-8ce0-95a096e79e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:03.519017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b0cf3-9b59-4980-8ce0-95a096e79e62 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:03.519300 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b0cf3-9b59-4980-8ce0-95a096e79e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:03.519580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b0cf3-9b59-4980-8ce0-95a096e79e62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:03.533247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484b0cf3-9b59-4980-8ce0-95a096e79e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:03.535108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484b0cf3-9b59-4980-8ce0-95a096e79e62 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:03.553602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-484b0cf3-9b59-4980-8ce0-95a096e79e62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:03.553602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:03.553602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:03.553602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:03.553602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:03.553602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:03.553602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:03.553602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:03.553602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:03.553602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:03.553602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:03.553602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:03.553602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:51:03.553602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:51:03.553602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:51:03.555136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:51:03.555136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:03.555136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:03.555136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:03.555136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:03.555136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:03.555136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:03.555136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:03.555136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:03.555136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:03.555136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:03.555136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:03.555136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:03.555136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:03.555136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:03.555136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:03.555136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:03.556214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:51:03.556214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:51:03.556214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:03.556214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:03.556214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 618/1224] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:51:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:03.561560 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e9964-ceef-4591-ba78-a43961e54cd6 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:03.673586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e9964-ceef-4591-ba78-a43961e54cd6 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:03.708334 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009fc1f-8182-4668-81d0-3185a694f51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:03.709087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009fc1f-8182-4668-81d0-3185a694f51d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:03.709381 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009fc1f-8182-4668-81d0-3185a694f51d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:03.709709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1009fc1f-8182-4668-81d0-3185a694f51d None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:03.710442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009fc1f-8182-4668-81d0-3185a694f51d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:03.710824 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009fc1f-8182-4668-81d0-3185a694f51d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:03.712297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1009fc1f-8182-4668-81d0-3185a694f51d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:03.725969 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 607/1225] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:51:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:03.732239 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b0cf3-9b59-4980-8ce0-95a096e79e62 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:03.835528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:03.843540 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484b0cf3-9b59-4980-8ce0-95a096e79e62 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:03.881988 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643cb22e-87e5-438f-b5b8-e40bb8b6ea3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:03.882837 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cb22e-87e5-438f-b5b8-e40bb8b6ea3f None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:03.883129 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cb22e-87e5-438f-b5b8-e40bb8b6ea3f None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:03.883409 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cb22e-87e5-438f-b5b8-e40bb8b6ea3f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:03.892367 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643cb22e-87e5-438f-b5b8-e40bb8b6ea3f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:03.893909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643cb22e-87e5-438f-b5b8-e40bb8b6ea3f None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:03.909733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-643cb22e-87e5-438f-b5b8-e40bb8b6ea3f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 22:51:03.909733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:03.909733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:03.909733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:03.909733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:03.909733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:03.909733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:03.909733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:03.909733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:03.909733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:03.909733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:03.909733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:03.909733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:51:03.909733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:51:03.909733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:51:03.909733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:51:03.910948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:03.910948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:03.910948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:51:03.910948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:51:03.910948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:51:03.910948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:51:03.910948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:03.910948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:03.910948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:03.910948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:03.910948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:03.910948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:03.910948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:03.910948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:03.910948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:03.910948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:03.910948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:03.910948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:03.912134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:03.912134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:03.912134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:51:03.912134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:51:03.912134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:51:03.912134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:51:03.912134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:51:03.912134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:51:03.912134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 22:51:03.912134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:03.912134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 619/1226] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:51:03 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:51:03.916949 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009fc1f-8182-4668-81d0-3185a694f51d None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:03.967683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:04.027784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1009fc1f-8182-4668-81d0-3185a694f51d None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:04.067437 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc919b3-614e-4b79-b2ab-333815d78fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:04.068296 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc919b3-614e-4b79-b2ab-333815d78fbc None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:04.068694 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc919b3-614e-4b79-b2ab-333815d78fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:04.069119 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc919b3-614e-4b79-b2ab-333815d78fbc None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:04.070236 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc919b3-614e-4b79-b2ab-333815d78fbc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:04.070562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc919b3-614e-4b79-b2ab-333815d78fbc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:04.072364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc919b3-614e-4b79-b2ab-333815d78fbc None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:04.092790 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 608/1227] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jun 16 22:51:03 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:04.098940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643cb22e-87e5-438f-b5b8-e40bb8b6ea3f None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:04.212095 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643cb22e-87e5-438f-b5b8-e40bb8b6ea3f None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:04.249585 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51c2b7-bd83-49d2-987b-815a011792b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:04.250382 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51c2b7-bd83-49d2-987b-815a011792b6 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:04.250695 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51c2b7-bd83-49d2-987b-815a011792b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:04.250977 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51c2b7-bd83-49d2-987b-815a011792b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:04.263751 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51c2b7-bd83-49d2-987b-815a011792b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:04.265180 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f51c2b7-bd83-49d2-987b-815a011792b6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:04.283681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-0f51c2b7-bd83-49d2-987b-815a011792b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:04.283681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:04.283681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:04.283681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:04.283681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:04.283681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:04.283681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:04.283681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:04.283681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:04.283681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:04.283681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:04.283681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:04.283681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:51:04.283681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:51:04.283681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:51:04.284760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:51:04.284760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:04.284760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:04.284760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:04.284760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:04.284760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:04.284760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:04.284760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:04.284760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:04.284760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:04.284760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:04.284760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:04.284760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:04.284760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:04.284760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:04.284760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:04.284760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:04.285977 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:51:04.285977 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:51:04.285977 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:04.285977 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:04.285977 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 620/1228] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:51:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:04.290499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc919b3-614e-4b79-b2ab-333815d78fbc None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:04.399521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc919b3-614e-4b79-b2ab-333815d78fbc None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:04.402365 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:04.434585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b14232-8c73-42f7-8c48-3519f7d0c112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:04.435289 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b14232-8c73-42f7-8c48-3519f7d0c112 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:04.435580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b14232-8c73-42f7-8c48-3519f7d0c112 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:04.435885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b14232-8c73-42f7-8c48-3519f7d0c112 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:04.436587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b14232-8c73-42f7-8c48-3519f7d0c112 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:04.436902 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b14232-8c73-42f7-8c48-3519f7d0c112 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:04.438318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b14232-8c73-42f7-8c48-3519f7d0c112 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:04.450680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 609/1229] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:51:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:04.456391 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51c2b7-bd83-49d2-987b-815a011792b6 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:04.466652 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:04.572141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51c2b7-bd83-49d2-987b-815a011792b6 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:04.614946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355a4d6-f748-42db-8200-1fd295ea6d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:04.615742 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355a4d6-f748-42db-8200-1fd295ea6d7e None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:04.616025 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355a4d6-f748-42db-8200-1fd295ea6d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:04.616303 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e355a4d6-f748-42db-8200-1fd295ea6d7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:04.623938 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355a4d6-f748-42db-8200-1fd295ea6d7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:04.625654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e355a4d6-f748-42db-8200-1fd295ea6d7e None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:04.644924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-e355a4d6-f748-42db-8200-1fd295ea6d7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:51:04.644924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:04.644924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:04.644924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:04.644924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:04.644924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:04.644924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:04.644924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:04.644924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:04.644924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:04.644924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:04.644924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:04.644924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:51:04.644924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:51:04.644924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:51:04.645990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:51:04.645990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:04.645990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:04.645990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:04.645990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:04.645990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:04.645990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:04.645990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:04.645990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:04.645990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:04.645990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:04.645990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:04.645990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:04.645990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:04.645990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:04.645990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:04.645990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:04.647105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:51:04.647105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:51:04.647105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:51:04.647105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:51:04.647105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:51:04.647105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:51:04.647105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:51:04.647105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:04.647105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 621/1230] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:51:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:04.652409 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b14232-8c73-42f7-8c48-3519f7d0c112 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:04.769287 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b14232-8c73-42f7-8c48-3519f7d0c112 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:04.805782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032d5aa8-b9d5-4768-8834-a0bf21f48815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:04.806446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d5aa8-b9d5-4768-8834-a0bf21f48815 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:04.806778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d5aa8-b9d5-4768-8834-a0bf21f48815 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:04.807061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032d5aa8-b9d5-4768-8834-a0bf21f48815 None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:04.809291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d5aa8-b9d5-4768-8834-a0bf21f48815 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:04.809613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d5aa8-b9d5-4768-8834-a0bf21f48815 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:04.811201 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032d5aa8-b9d5-4768-8834-a0bf21f48815 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:04.823966 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 610/1231] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:51:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:51:04.829687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355a4d6-f748-42db-8200-1fd295ea6d7e None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:04.900834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:04.938131 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e355a4d6-f748-42db-8200-1fd295ea6d7e None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:04.973183 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c58343-a2a1-484c-91c7-9cbc717cece0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:04.974197 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c58343-a2a1-484c-91c7-9cbc717cece0 None admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:04.974490 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c58343-a2a1-484c-91c7-9cbc717cece0 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:04.974796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c58343-a2a1-484c-91c7-9cbc717cece0 None admin] PATH_INFO: `/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/f9593939641c4c7797c1981206b40e7d/roles/ec841a43f9924fa1b2209bc5ecea5e04` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:04.996503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c58343-a2a1-484c-91c7-9cbc717cece0 None admin] RBAC: Authorizing `identity:create_grant(project_id=bc48914270df4db996bd2a4b5b9fc777, user_id=f9593939641c4c7797c1981206b40e7d, role_id=ec841a43f9924fa1b2209bc5ecea5e04)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:04.997958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c58343-a2a1-484c-91c7-9cbc717cece0 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:05.015688 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 622/1232] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:51:04 2022] PUT /identity/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/f9593939641c4c7797c1981206b40e7d/roles/ec841a43f9924fa1b2209bc5ecea5e04 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:51:07.252206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef2d6b-4bf5-461f-b012-b7452d31e6b9 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:07.252730 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef2d6b-4bf5-461f-b012-b7452d31e6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:07.252836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ef2d6b-4bf5-461f-b012-b7452d31e6b9 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:07.254140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 611/1233] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:51:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:51:07.259830 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18023d6-c19a-419d-8808-152a46998424 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:07.260255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18023d6-c19a-419d-8808-152a46998424 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:07.260955 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18023d6-c19a-419d-8808-152a46998424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:07.277681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a18023d6-c19a-419d-8808-152a46998424 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:51:07.278889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a18023d6-c19a-419d-8808-152a46998424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:51:07.353252 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 623/1234] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:51:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:51:07.378019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef2d6b-4bf5-461f-b012-b7452d31e6b9 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:07.406479 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:07.492811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ef2d6b-4bf5-461f-b012-b7452d31e6b9 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:07.527648 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9963f-6836-4cb5-88ec-5587ff3f2163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:07.528448 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9963f-6836-4cb5-88ec-5587ff3f2163 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:07.528732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9963f-6836-4cb5-88ec-5587ff3f2163 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:07.529015 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c9963f-6836-4cb5-88ec-5587ff3f2163 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:07.539937 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c9963f-6836-4cb5-88ec-5587ff3f2163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:51:07.548042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-c2c9963f-6836-4cb5-88ec-5587ff3f2163 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:51:07.548042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:07.548042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:07.548042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:07.548042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:07.548042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:07.548042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:07.548042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:07.548042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:07.548042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:07.548042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:07.548042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:07.548042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:51:07.548042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:51:07.548042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:51:07.548042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:51:07.549153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:51:07.549153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:51:07.549153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:51:07.549153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:51:07.549153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:07.549153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:07.549153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:07.549153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:07.549153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:07.549153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:07.549153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:07.549153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:07.549153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:07.549153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:07.549153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:07.549153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:07.550172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:07.550172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:07.550172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:07.550172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:07.550172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:51:07.550172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:51:07.550172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:51:07.550172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:51:07.550172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:51:07.550172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:51:07.550172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 22:51:07.550172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:51:07.550172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 612/1235] 10.222.0.17 () {56 vars in 1102 bytes} [Thu Jun 16 22:51:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:07.555509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18023d6-c19a-419d-8808-152a46998424 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:07.604708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:07.661601 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18023d6-c19a-419d-8808-152a46998424 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:07.697356 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0a3b4-185e-417d-92fd-50b8925ae9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:07.698529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0a3b4-185e-417d-92fd-50b8925ae9d7 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:07.698850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0a3b4-185e-417d-92fd-50b8925ae9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:07.699136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd0a3b4-185e-417d-92fd-50b8925ae9d7 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:07.699873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0a3b4-185e-417d-92fd-50b8925ae9d7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:07.700182 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0a3b4-185e-417d-92fd-50b8925ae9d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:07.701609 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bd0a3b4-185e-417d-92fd-50b8925ae9d7 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:07.711109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 624/1236] 10.222.0.17 () {56 vars in 1103 bytes} [Thu Jun 16 22:51:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:07.717603 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9963f-6836-4cb5-88ec-5587ff3f2163 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:07.811792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:07.828158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c9963f-6836-4cb5-88ec-5587ff3f2163 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:07.863285 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edb5cd-0a62-4326-93ef-aff12d3ac28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:07.864120 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edb5cd-0a62-4326-93ef-aff12d3ac28f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:07.864420 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edb5cd-0a62-4326-93ef-aff12d3ac28f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:07.864704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edb5cd-0a62-4326-93ef-aff12d3ac28f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:07.880324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22edb5cd-0a62-4326-93ef-aff12d3ac28f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:07.881829 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22edb5cd-0a62-4326-93ef-aff12d3ac28f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:07.902576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-22edb5cd-0a62-4326-93ef-aff12d3ac28f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:07.902576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:07.902576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:07.902576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:07.902576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:07.902576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:07.902576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:07.902576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:07.902576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:07.902576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:07.902576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:07.902576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:07.902576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:51:07.902576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:51:07.902576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:51:07.903614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:51:07.903614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:07.903614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:07.903614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:07.903614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:07.903614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:07.903614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:07.903614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:07.903614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:07.903614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:07.903614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:07.903614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:07.903614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:07.903614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:07.903614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:07.903614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:07.903614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:07.904544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:51:07.904544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:51:07.904544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:07.904544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:51:07.904544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 613/1237] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:51:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:51:07.910235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0a3b4-185e-417d-92fd-50b8925ae9d7 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:08.019791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd0a3b4-185e-417d-92fd-50b8925ae9d7 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:08.032001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:08.063731 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f9b53-7058-426d-a6c4-e0f4d98545a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:08.064283 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f9b53-7058-426d-a6c4-e0f4d98545a2 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:08.064581 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f9b53-7058-426d-a6c4-e0f4d98545a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:08.064863 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f9b53-7058-426d-a6c4-e0f4d98545a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:08.065587 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f9b53-7058-426d-a6c4-e0f4d98545a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:08.065908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f9b53-7058-426d-a6c4-e0f4d98545a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:08.067384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f9b53-7058-426d-a6c4-e0f4d98545a2 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:08.079939 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 625/1238] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:51:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:08.085617 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edb5cd-0a62-4326-93ef-aff12d3ac28f None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:08.190310 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edb5cd-0a62-4326-93ef-aff12d3ac28f None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:08.231331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffa8d5-a51a-41ee-bb20-6ca5cf70cbc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:08.232429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffa8d5-a51a-41ee-bb20-6ca5cf70cbc5 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:08.232712 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffa8d5-a51a-41ee-bb20-6ca5cf70cbc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:08.232987 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ffa8d5-a51a-41ee-bb20-6ca5cf70cbc5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:08.236644 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:08.242205 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffa8d5-a51a-41ee-bb20-6ca5cf70cbc5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:08.243990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ffa8d5-a51a-41ee-bb20-6ca5cf70cbc5 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:08.258941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-f6ffa8d5-a51a-41ee-bb20-6ca5cf70cbc5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 22:51:08.258941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:08.258941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:08.258941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:08.258941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:08.258941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:08.258941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:08.258941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:08.258941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:08.258941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:08.258941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:08.258941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:08.258941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:51:08.258941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:51:08.258941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:51:08.258941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:51:08.260586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:08.260586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:08.260586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:51:08.260586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:51:08.260586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:51:08.260586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:51:08.260586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:08.260586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:08.260586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:08.260586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:08.260586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:08.260586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:08.260586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:08.260586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:08.260586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:08.260586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:08.260586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:08.260586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:08.261755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:08.261755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:08.261755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:51:08.261755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:51:08.261755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:51:08.261755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:51:08.261755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:51:08.261755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:51:08.261755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 22:51:08.261755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:51:08.261755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 614/1239] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:51:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:08.266150 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f9b53-7058-426d-a6c4-e0f4d98545a2 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:08.377060 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f9b53-7058-426d-a6c4-e0f4d98545a2 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:08.414875 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b945226-1fb3-487b-8b6a-593e9650bb22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:08.415598 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b945226-1fb3-487b-8b6a-593e9650bb22 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:08.415933 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b945226-1fb3-487b-8b6a-593e9650bb22 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:08.416225 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b945226-1fb3-487b-8b6a-593e9650bb22 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:08.417292 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b945226-1fb3-487b-8b6a-593e9650bb22 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:08.417679 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b945226-1fb3-487b-8b6a-593e9650bb22 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:08.419372 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b945226-1fb3-487b-8b6a-593e9650bb22 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:08.439014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 626/1240] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jun 16 22:51:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:08.444535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffa8d5-a51a-41ee-bb20-6ca5cf70cbc5 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:08.558707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ffa8d5-a51a-41ee-bb20-6ca5cf70cbc5 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:08.601132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0b0a7-15b3-4d3c-bc36-a0e34ce3c505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:08.602014 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0b0a7-15b3-4d3c-bc36-a0e34ce3c505 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:08.602309 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0b0a7-15b3-4d3c-bc36-a0e34ce3c505 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:08.602610 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a0b0a7-15b3-4d3c-bc36-a0e34ce3c505 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:08.667942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:08.675109 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a0b0a7-15b3-4d3c-bc36-a0e34ce3c505 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:08.676633 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a0b0a7-15b3-4d3c-bc36-a0e34ce3c505 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:08.789862 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 615/1241] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:51:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:51:08.796632 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b945226-1fb3-487b-8b6a-593e9650bb22 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:08.828954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:08.905329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b945226-1fb3-487b-8b6a-593e9650bb22 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:08.949917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5233e37-dee0-4186-ab00-82ca824c2f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:08.950810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5233e37-dee0-4186-ab00-82ca824c2f66 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:08.951069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5233e37-dee0-4186-ab00-82ca824c2f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:08.951345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5233e37-dee0-4186-ab00-82ca824c2f66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:08.964126 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5233e37-dee0-4186-ab00-82ca824c2f66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:08.965573 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5233e37-dee0-4186-ab00-82ca824c2f66 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:08.988329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-b5233e37-dee0-4186-ab00-82ca824c2f66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:08.988329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:08.988329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:08.988329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:08.988329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:08.988329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:08.988329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:08.988329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:08.988329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:08.988329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:08.988329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:08.988329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:08.988329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:51:08.988329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:51:08.988329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:51:08.989302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:51:08.989302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:08.989302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:08.989302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:08.989302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:08.989302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:08.989302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:08.989302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:08.989302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:08.989302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:08.989302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:08.989302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:08.989302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:08.989302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:08.989302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:08.989302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:08.989302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:08.990256 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:51:08.990256 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:51:08.990256 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:08.990256 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:08.990256 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 627/1242] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:51:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:51:08.998488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0b0a7-15b3-4d3c-bc36-a0e34ce3c505 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:09.115698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:09.124845 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a0b0a7-15b3-4d3c-bc36-a0e34ce3c505 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:09.162096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4ed97-dc69-4033-bcff-dbbd7308f115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:09.162801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4ed97-dc69-4033-bcff-dbbd7308f115 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:09.163092 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4ed97-dc69-4033-bcff-dbbd7308f115 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:09.163490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff4ed97-dc69-4033-bcff-dbbd7308f115 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:09.164485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4ed97-dc69-4033-bcff-dbbd7308f115 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:09.164830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4ed97-dc69-4033-bcff-dbbd7308f115 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:09.166744 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff4ed97-dc69-4033-bcff-dbbd7308f115 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:09.180738 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 616/1243] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:51:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:09.186702 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5233e37-dee0-4186-ab00-82ca824c2f66 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:09.271187 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:09.295400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5233e37-dee0-4186-ab00-82ca824c2f66 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:09.332321 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b5454-08f8-4318-a267-c8d85de4b4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:09.333135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b5454-08f8-4318-a267-c8d85de4b4c8 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:09.333424 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b5454-08f8-4318-a267-c8d85de4b4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:09.333812 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b5454-08f8-4318-a267-c8d85de4b4c8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:09.341541 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b5454-08f8-4318-a267-c8d85de4b4c8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:09.343064 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b5454-08f8-4318-a267-c8d85de4b4c8 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:09.357400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-cf6b5454-08f8-4318-a267-c8d85de4b4c8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:51:09.357400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:09.357400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:09.357400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:09.357400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:09.357400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:09.357400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:09.357400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:09.357400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:09.357400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:09.357400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:09.357400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:09.357400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:51:09.357400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:51:09.357400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:51:09.358519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:51:09.358519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:09.358519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:09.358519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:09.358519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:09.358519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:09.358519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:09.358519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:09.358519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:09.358519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:09.358519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:09.358519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:09.358519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:09.358519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:09.358519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:09.358519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:09.358519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:09.359638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:51:09.359638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:51:09.359638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:51:09.359638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:51:09.359638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:51:09.359638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:51:09.359638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:51:09.359638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:09.359638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 628/1244] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:51:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:09.364634 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4ed97-dc69-4033-bcff-dbbd7308f115 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:09.472282 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff4ed97-dc69-4033-bcff-dbbd7308f115 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:09.508864 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28d9773-3f72-44e6-8a3e-bb102db2063d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:09.509577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d9773-3f72-44e6-8a3e-bb102db2063d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:09.509905 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d9773-3f72-44e6-8a3e-bb102db2063d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:09.510254 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28d9773-3f72-44e6-8a3e-bb102db2063d None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:09.511758 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d9773-3f72-44e6-8a3e-bb102db2063d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:09.512069 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d9773-3f72-44e6-8a3e-bb102db2063d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:09.513687 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28d9773-3f72-44e6-8a3e-bb102db2063d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:09.527935 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 617/1245] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:51:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:09.533813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b5454-08f8-4318-a267-c8d85de4b4c8 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:09.644331 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b5454-08f8-4318-a267-c8d85de4b4c8 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:09.681264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a1523fe-d3fc-4caa-8116-10c75c061359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:09.681929 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1523fe-d3fc-4caa-8116-10c75c061359 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:09.682217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1523fe-d3fc-4caa-8116-10c75c061359 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:09.682501 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a1523fe-d3fc-4caa-8116-10c75c061359 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:09.683361 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1523fe-d3fc-4caa-8116-10c75c061359 None admin] RBAC: Adding query filter params (role.id=ec841a43f9924fa1b2209bc5ecea5e04, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=f9593939641c4c7797c1981206b40e7d) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:09.683681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1523fe-d3fc-4caa-8116-10c75c061359 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:09.685133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a1523fe-d3fc-4caa-8116-10c75c061359 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:09.691226 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 629/1246] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:51:09 2022] GET /identity/v3/role_assignments?user.id=f9593939641c4c7797c1981206b40e7d&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=ec841a43f9924fa1b2209bc5ecea5e04 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:51:11.941120 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c24f85-e53c-421a-8ddf-f448fede5973 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:11.941664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c24f85-e53c-421a-8ddf-f448fede5973 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:11.941744 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c24f85-e53c-421a-8ddf-f448fede5973 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:11.943256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 618/1247] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:51:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:51:11.948336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e00c8b-27ca-483c-ad4f-cb62eb6cd5cc None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:11.948795 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e00c8b-27ca-483c-ad4f-cb62eb6cd5cc None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:11.949103 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e00c8b-27ca-483c-ad4f-cb62eb6cd5cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:11.955146 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:11.968909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-41e00c8b-27ca-483c-ad4f-cb62eb6cd5cc None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:51:11.970116 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-41e00c8b-27ca-483c-ad4f-cb62eb6cd5cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:51:12.048585 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 630/1248] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:51:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:51:12.079945 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c24f85-e53c-421a-8ddf-f448fede5973 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:12.101442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:12.200523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c24f85-e53c-421a-8ddf-f448fede5973 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:12.247038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d1d62-5d2a-40bf-b628-67aec14e4e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:12.248034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d1d62-5d2a-40bf-b628-67aec14e4e91 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:12.248395 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d1d62-5d2a-40bf-b628-67aec14e4e91 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:12.248739 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0d1d62-5d2a-40bf-b628-67aec14e4e91 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:12.260695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0d1d62-5d2a-40bf-b628-67aec14e4e91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:51:12.269050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-1b0d1d62-5d2a-40bf-b628-67aec14e4e91 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:51:12.269050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:12.269050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:12.269050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:12.269050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:12.269050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:12.269050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:12.269050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:12.269050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:12.269050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:12.269050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:12.269050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:12.269050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:51:12.269050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:51:12.269050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:51:12.269050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:51:12.270019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:51:12.270019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:51:12.270019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:51:12.270019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:51:12.270019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:12.270019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:12.270019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:12.270019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:12.270019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:12.270019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:12.270019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:12.270019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:12.270019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:12.270019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:12.270019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:12.270019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:12.270893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:12.270893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:12.270893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:12.270893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:12.270893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:51:12.270893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:51:12.270893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:51:12.270893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:51:12.270893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:51:12.270893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:51:12.270893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:51:12.270893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:51:12.270893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 619/1249] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:51:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:51:12.277478 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e00c8b-27ca-483c-ad4f-cb62eb6cd5cc None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:12.391561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e00c8b-27ca-483c-ad4f-cb62eb6cd5cc None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:12.427682 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:12.434721 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b3910-9138-4b1e-a9a0-a4a05927ec04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:12.436735 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b3910-9138-4b1e-a9a0-a4a05927ec04 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:12.436735 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b3910-9138-4b1e-a9a0-a4a05927ec04 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:12.436735 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48b3910-9138-4b1e-a9a0-a4a05927ec04 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:12.436735 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b3910-9138-4b1e-a9a0-a4a05927ec04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:12.437170 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b3910-9138-4b1e-a9a0-a4a05927ec04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:12.438354 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48b3910-9138-4b1e-a9a0-a4a05927ec04 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:12.447163 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 631/1250] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:51:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:51:12.453874 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d1d62-5d2a-40bf-b628-67aec14e4e91 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:12.539754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:12.562516 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0d1d62-5d2a-40bf-b628-67aec14e4e91 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:12.600611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c6f8b-6218-4301-b779-aef36f7455b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:12.601402 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c6f8b-6218-4301-b779-aef36f7455b2 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:12.601689 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c6f8b-6218-4301-b779-aef36f7455b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:12.601981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c6f8b-6218-4301-b779-aef36f7455b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:12.613248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c6f8b-6218-4301-b779-aef36f7455b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:12.614716 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c6f8b-6218-4301-b779-aef36f7455b2 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:12.632260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-000c6f8b-6218-4301-b779-aef36f7455b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:12.632260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:12.632260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:12.632260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:12.632260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:12.632260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:12.632260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:12.632260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:12.632260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:12.632260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:12.632260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:12.632260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:12.632260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:51:12.632260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:51:12.632260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:51:12.633353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:51:12.633353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:12.633353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:12.633353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:12.633353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:12.633353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:12.633353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:12.633353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:12.633353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:12.633353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:12.633353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:12.633353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:12.633353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:12.633353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:12.633353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:12.633353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:12.633353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:12.634450 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:51:12.634450 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:51:12.634450 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:12.634450 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:51:12.634450 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 620/1251] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:12.639368 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b3910-9138-4b1e-a9a0-a4a05927ec04 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:12.749877 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48b3910-9138-4b1e-a9a0-a4a05927ec04 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:12.786649 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcfb7fa-b542-4604-9a69-d80de5ddb42a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:12.787361 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfb7fa-b542-4604-9a69-d80de5ddb42a None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:12.787646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfb7fa-b542-4604-9a69-d80de5ddb42a None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:12.787922 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afcfb7fa-b542-4604-9a69-d80de5ddb42a None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:12.788732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcfb7fa-b542-4604-9a69-d80de5ddb42a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:12.789080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcfb7fa-b542-4604-9a69-d80de5ddb42a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:12.790706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afcfb7fa-b542-4604-9a69-d80de5ddb42a None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:12.808108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 632/1252] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:51:12.813637 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c6f8b-6218-4301-b779-aef36f7455b2 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:12.925019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c6f8b-6218-4301-b779-aef36f7455b2 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:12.959114 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:12.969883 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd3056-75af-44bc-924b-492894f6af6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:12.970959 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd3056-75af-44bc-924b-492894f6af6f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:12.971419 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd3056-75af-44bc-924b-492894f6af6f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:12.971897 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5dd3056-75af-44bc-924b-492894f6af6f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:12.983190 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd3056-75af-44bc-924b-492894f6af6f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:12.985177 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5dd3056-75af-44bc-924b-492894f6af6f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:13.005267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-b5dd3056-75af-44bc-924b-492894f6af6f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 22:51:13.005267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:13.005267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:13.005267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:13.005267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:13.005267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:13.005267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:13.005267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:13.005267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:13.005267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:13.005267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:13.005267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:13.005267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:51:13.005267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:51:13.005267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:51:13.005267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:51:13.006291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:13.006291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:13.006291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:51:13.006291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:51:13.006291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:51:13.006291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:51:13.006291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:13.006291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:13.006291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:13.006291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:13.006291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:13.006291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:13.006291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:13.006291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:13.006291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:13.006291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:13.006291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:13.006291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:13.007286 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:13.007286 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:13.007286 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:51:13.007286 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:51:13.007286 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:51:13.007286 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:51:13.007286 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:51:13.007286 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:51:13.007286 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 22:51:13.007286 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:51:13.007286 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 621/1253] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:51:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:13.013360 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcfb7fa-b542-4604-9a69-d80de5ddb42a None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:13.077584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:13.123582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afcfb7fa-b542-4604-9a69-d80de5ddb42a None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:13.159805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09605b-5aa1-4831-9277-e59e7cb7e3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:13.160484 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09605b-5aa1-4831-9277-e59e7cb7e3e7 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:13.160774 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09605b-5aa1-4831-9277-e59e7cb7e3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:13.161054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09605b-5aa1-4831-9277-e59e7cb7e3e7 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:13.161890 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09605b-5aa1-4831-9277-e59e7cb7e3e7 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:13.162204 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09605b-5aa1-4831-9277-e59e7cb7e3e7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:13.163693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09605b-5aa1-4831-9277-e59e7cb7e3e7 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:13.185507 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 633/1254] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jun 16 22:51:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:13.191295 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd3056-75af-44bc-924b-492894f6af6f None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:13.313901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5dd3056-75af-44bc-924b-492894f6af6f None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:13.348677 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e109c5-e232-4997-9830-e5e5caa0f1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:13.349333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e109c5-e232-4997-9830-e5e5caa0f1e2 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:13.349625 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e109c5-e232-4997-9830-e5e5caa0f1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:13.349909 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e109c5-e232-4997-9830-e5e5caa0f1e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:13.408239 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:13.422472 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e109c5-e232-4997-9830-e5e5caa0f1e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:13.423957 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e109c5-e232-4997-9830-e5e5caa0f1e2 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:13.533975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 622/1255] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:51:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:51:13.539799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09605b-5aa1-4831-9277-e59e7cb7e3e7 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:13.661043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09605b-5aa1-4831-9277-e59e7cb7e3e7 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:13.676002 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:13.699636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9e245-009c-4186-a6fc-8d57dbd23502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:13.700534 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9e245-009c-4186-a6fc-8d57dbd23502 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:13.700854 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9e245-009c-4186-a6fc-8d57dbd23502 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:13.701169 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c9e245-009c-4186-a6fc-8d57dbd23502 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:13.711764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c9e245-009c-4186-a6fc-8d57dbd23502 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:13.713903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c9e245-009c-4186-a6fc-8d57dbd23502 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:13.732674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-97c9e245-009c-4186-a6fc-8d57dbd23502 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:13.732674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:13.732674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:13.732674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:13.732674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:13.732674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:13.732674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:13.732674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:13.732674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:13.732674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:13.732674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:13.732674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:13.732674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:51:13.732674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:51:13.732674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:51:13.733759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:51:13.733759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:13.733759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:13.733759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:13.733759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:13.733759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:13.733759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:13.733759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:13.733759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:13.733759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:13.733759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:13.733759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:13.733759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:13.733759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:13.733759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:13.733759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:13.733759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:13.734684 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:51:13.734684 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:51:13.734684 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:13.734684 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:13.734684 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 634/1256] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:51:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:13.739624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e109c5-e232-4997-9830-e5e5caa0f1e2 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:13.832973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:13.846852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e109c5-e232-4997-9830-e5e5caa0f1e2 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:13.884202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601883da-bd02-4f11-b522-46cd002c81a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:13.884896 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601883da-bd02-4f11-b522-46cd002c81a2 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:13.885184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601883da-bd02-4f11-b522-46cd002c81a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:13.885970 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601883da-bd02-4f11-b522-46cd002c81a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:13.886751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601883da-bd02-4f11-b522-46cd002c81a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:13.887067 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601883da-bd02-4f11-b522-46cd002c81a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:13.888500 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601883da-bd02-4f11-b522-46cd002c81a2 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:13.908622 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 623/1257] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:51:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:13.914643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9e245-009c-4186-a6fc-8d57dbd23502 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:14.033843 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c9e245-009c-4186-a6fc-8d57dbd23502 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:14.071696 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79ebd4-0c5c-44d2-98ad-fe91b59056a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:14.072505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79ebd4-0c5c-44d2-98ad-fe91b59056a4 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:14.073319 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79ebd4-0c5c-44d2-98ad-fe91b59056a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:14.073603 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d79ebd4-0c5c-44d2-98ad-fe91b59056a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:14.081217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79ebd4-0c5c-44d2-98ad-fe91b59056a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:14.082703 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d79ebd4-0c5c-44d2-98ad-fe91b59056a4 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:14.097686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-7d79ebd4-0c5c-44d2-98ad-fe91b59056a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:51:14.097686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:14.097686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:14.097686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:14.097686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:14.097686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:14.097686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:14.097686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:14.097686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:14.097686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:14.097686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:14.097686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:14.097686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:51:14.097686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:51:14.097686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:51:14.098719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:51:14.098719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:14.098719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:14.098719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:14.098719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:14.098719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:14.098719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:14.098719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:14.098719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:14.098719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:14.098719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:14.098719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:14.098719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:14.098719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:14.098719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:14.098719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:14.098719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:14.099624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:51:14.099624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:51:14.099624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:51:14.099624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:51:14.099624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:51:14.099624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:51:14.099624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:51:14.099624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:14.099624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 635/1258] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:51:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:14.105607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601883da-bd02-4f11-b522-46cd002c81a2 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:14.208869 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601883da-bd02-4f11-b522-46cd002c81a2 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:14.248163 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19030703-dac4-4a34-a9bb-2fd138b85c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:14.248843 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19030703-dac4-4a34-a9bb-2fd138b85c1c None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:14.249132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19030703-dac4-4a34-a9bb-2fd138b85c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:14.249415 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19030703-dac4-4a34-a9bb-2fd138b85c1c None admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:14.250209 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19030703-dac4-4a34-a9bb-2fd138b85c1c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:14.250526 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19030703-dac4-4a34-a9bb-2fd138b85c1c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:14.252026 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19030703-dac4-4a34-a9bb-2fd138b85c1c None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:14.264894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 624/1259] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:51:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:14.271176 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79ebd4-0c5c-44d2-98ad-fe91b59056a4 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:14.300374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:14.377459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d79ebd4-0c5c-44d2-98ad-fe91b59056a4 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:14.413212 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ce1175-e8f6-4cf9-888a-0f39c9ffbebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:14.413877 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce1175-e8f6-4cf9-888a-0f39c9ffbebc None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:14.414166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce1175-e8f6-4cf9-888a-0f39c9ffbebc None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:14.414448 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ce1175-e8f6-4cf9-888a-0f39c9ffbebc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:14.415331 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce1175-e8f6-4cf9-888a-0f39c9ffbebc None admin] RBAC: Adding query filter params (role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=f9593939641c4c7797c1981206b40e7d) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:14.415650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce1175-e8f6-4cf9-888a-0f39c9ffbebc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:14.417688 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ce1175-e8f6-4cf9-888a-0f39c9ffbebc None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:14.423320 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 636/1260] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:51:14 2022] GET /identity/v3/role_assignments?user.id=f9593939641c4c7797c1981206b40e7d&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:16.493026 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3927f98-1720-44ec-b6b0-ce06f2804583 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:16.493519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3927f98-1720-44ec-b6b0-ce06f2804583 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:16.493519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3927f98-1720-44ec-b6b0-ce06f2804583 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:16.495511 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 625/1261] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:51:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:51:16.500463 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57814c8e-bfe5-4731-981c-59acf4379b4e None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:16.500876 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57814c8e-bfe5-4731-981c-59acf4379b4e None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:16.501019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57814c8e-bfe5-4731-981c-59acf4379b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:16.517342 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-57814c8e-bfe5-4731-981c-59acf4379b4e None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:51:16.518500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-57814c8e-bfe5-4731-981c-59acf4379b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:51:16.588181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 637/1262] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:51:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:51:16.611585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3927f98-1720-44ec-b6b0-ce06f2804583 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:16.622566 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:16.711173 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3927f98-1720-44ec-b6b0-ce06f2804583 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:16.744440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0269b-c45e-451b-9f78-e1ed7f1d19b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:16.745170 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0269b-c45e-451b-9f78-e1ed7f1d19b2 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:16.745519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0269b-c45e-451b-9f78-e1ed7f1d19b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:16.745802 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e0269b-c45e-451b-9f78-e1ed7f1d19b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:16.758011 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e0269b-c45e-451b-9f78-e1ed7f1d19b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:51:16.765444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-80e0269b-c45e-451b-9f78-e1ed7f1d19b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:51:16.765444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:16.765444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:16.765444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:16.765444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:16.765444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:16.765444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:16.765444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:16.765444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:16.765444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:16.765444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:16.765444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:16.765444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:51:16.765444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:51:16.765444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:51:16.765444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:51:16.766627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:51:16.766627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:51:16.766627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:51:16.766627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:51:16.766627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:16.766627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:16.766627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:16.766627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:16.766627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:16.766627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:16.766627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:16.766627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:16.766627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:16.766627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:16.766627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:16.766627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:16.767932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:16.767932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:16.767932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:16.767932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:16.767932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:51:16.767932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:51:16.767932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:51:16.767932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:51:16.767932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:51:16.767932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:51:16.767932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:51:16.767932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:51:16.767932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 626/1263] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:51:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:16.772707 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57814c8e-bfe5-4731-981c-59acf4379b4e None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:16.791201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:16.870124 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57814c8e-bfe5-4731-981c-59acf4379b4e None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:16.904884 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ccf3b-5ba2-46d0-a93d-eb1f7526aa27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:16.905547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ccf3b-5ba2-46d0-a93d-eb1f7526aa27 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:16.905835 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ccf3b-5ba2-46d0-a93d-eb1f7526aa27 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:16.906109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76ccf3b-5ba2-46d0-a93d-eb1f7526aa27 None admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:16.906875 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ccf3b-5ba2-46d0-a93d-eb1f7526aa27 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:16.907183 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ccf3b-5ba2-46d0-a93d-eb1f7526aa27 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:16.908530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76ccf3b-5ba2-46d0-a93d-eb1f7526aa27 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:16.916365 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 638/1264] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:51:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:16.921970 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0269b-c45e-451b-9f78-e1ed7f1d19b2 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:16.986098 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:17.024431 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e0269b-c45e-451b-9f78-e1ed7f1d19b2 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:17.061228 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d3e0a-4ba1-4f0b-aa52-046e3cd2b8e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:17.062014 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d3e0a-4ba1-4f0b-aa52-046e3cd2b8e5 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:17.062294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d3e0a-4ba1-4f0b-aa52-046e3cd2b8e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:17.062568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7d3e0a-4ba1-4f0b-aa52-046e3cd2b8e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:17.073463 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d3e0a-4ba1-4f0b-aa52-046e3cd2b8e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:17.074916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7d3e0a-4ba1-4f0b-aa52-046e3cd2b8e5 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:17.091390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-2b7d3e0a-4ba1-4f0b-aa52-046e3cd2b8e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:17.091390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:17.091390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:17.091390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:17.091390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:17.091390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:17.091390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:17.091390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:17.091390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:17.091390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:17.091390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:17.091390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:17.091390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:51:17.091390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:51:17.091390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:51:17.092317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:51:17.092317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:17.092317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:17.092317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:17.092317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:17.092317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:17.092317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:17.092317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:17.092317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:17.092317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:17.092317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:17.092317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:17.092317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:17.092317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:17.092317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:17.092317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:17.092317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:17.093393 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:51:17.093393 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:51:17.093393 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:17.093393 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:51:17.093393 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 627/1265] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:17.098204 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ccf3b-5ba2-46d0-a93d-eb1f7526aa27 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:17.176766 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:17.195983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76ccf3b-5ba2-46d0-a93d-eb1f7526aa27 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:17.228143 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f5606-3689-471b-802a-c0cabf863d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:17.229363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f5606-3689-471b-802a-c0cabf863d69 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:17.229635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f5606-3689-471b-802a-c0cabf863d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:17.229911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f5606-3689-471b-802a-c0cabf863d69 None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:17.230662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f5606-3689-471b-802a-c0cabf863d69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:17.230976 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f5606-3689-471b-802a-c0cabf863d69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:17.232367 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367f5606-3689-471b-802a-c0cabf863d69 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:17.243847 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 639/1266] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:17.250013 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d3e0a-4ba1-4f0b-aa52-046e3cd2b8e5 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:17.344759 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7d3e0a-4ba1-4f0b-aa52-046e3cd2b8e5 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:17.363143 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:17.377093 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd0e7b-67f3-432f-81a0-28a6e3357aa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:17.377875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd0e7b-67f3-432f-81a0-28a6e3357aa8 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:17.378158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd0e7b-67f3-432f-81a0-28a6e3357aa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:17.378433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03dd0e7b-67f3-432f-81a0-28a6e3357aa8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:17.385602 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd0e7b-67f3-432f-81a0-28a6e3357aa8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:17.387052 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03dd0e7b-67f3-432f-81a0-28a6e3357aa8 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:17.401544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-03dd0e7b-67f3-432f-81a0-28a6e3357aa8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 22:51:17.401544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:17.401544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:17.401544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:17.401544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:17.401544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:17.401544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:17.401544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:17.401544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:17.401544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:17.401544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:17.401544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:17.401544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:51:17.401544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:51:17.401544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:51:17.401544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:51:17.402656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:17.402656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:17.402656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:51:17.402656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:51:17.402656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:51:17.402656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:51:17.402656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:17.402656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:17.402656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:17.402656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:17.402656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:17.402656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:17.402656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:17.402656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:17.402656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:17.402656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:17.402656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:17.402656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:17.403832 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:17.403832 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:17.403832 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:51:17.403832 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:51:17.403832 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:51:17.403832 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:51:17.403832 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:51:17.403832 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:51:17.403832 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 22:51:17.403832 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:51:17.403832 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 628/1267] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:51:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:17.408217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f5606-3689-471b-802a-c0cabf863d69 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:17.503547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367f5606-3689-471b-802a-c0cabf863d69 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:17.536231 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b63de2-71b6-4753-aa5f-df1cf36d5e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:17.536880 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b63de2-71b6-4753-aa5f-df1cf36d5e97 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:17.537166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b63de2-71b6-4753-aa5f-df1cf36d5e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:17.537445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b63de2-71b6-4753-aa5f-df1cf36d5e97 None admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:17.538264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b63de2-71b6-4753-aa5f-df1cf36d5e97 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:17.538576 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b63de2-71b6-4753-aa5f-df1cf36d5e97 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:17.540081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b63de2-71b6-4753-aa5f-df1cf36d5e97 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:17.548878 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:17.557278 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 640/1268] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jun 16 22:51:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:17.562688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd0e7b-67f3-432f-81a0-28a6e3357aa8 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:17.659855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03dd0e7b-67f3-432f-81a0-28a6e3357aa8 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:17.691540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa383327-eee7-43c0-9c0d-ba251c973084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:17.692318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa383327-eee7-43c0-9c0d-ba251c973084 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:17.692602 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa383327-eee7-43c0-9c0d-ba251c973084 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:17.692881 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa383327-eee7-43c0-9c0d-ba251c973084 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:17.703533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa383327-eee7-43c0-9c0d-ba251c973084 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:17.704955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa383327-eee7-43c0-9c0d-ba251c973084 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:17.721893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-fa383327-eee7-43c0-9c0d-ba251c973084 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:17.721893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:17.721893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:17.721893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:17.721893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:17.721893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:17.721893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:17.721893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:17.721893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:17.721893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:17.721893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:17.721893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:17.721893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:51:17.721893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:51:17.721893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:51:17.722931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:51:17.722931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:17.722931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:17.722931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:17.722931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:17.722931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:17.722931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:17.722931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:17.722931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:17.722931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:17.722931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:17.722931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:17.722931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:17.722931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:17.722931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:17.722931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:17.722931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:17.723985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:51:17.723985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:51:17.723985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:17.723985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:51:17.723985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 629/1269] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:17.728509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b63de2-71b6-4753-aa5f-df1cf36d5e97 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:17.828980 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b63de2-71b6-4753-aa5f-df1cf36d5e97 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:17.862057 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1652ac9-3e15-4c77-8630-225b01cc20ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:17.862741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1652ac9-3e15-4c77-8630-225b01cc20ea None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:17.863025 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1652ac9-3e15-4c77-8630-225b01cc20ea None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:17.863298 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1652ac9-3e15-4c77-8630-225b01cc20ea None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:17.864035 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1652ac9-3e15-4c77-8630-225b01cc20ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:17.864339 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1652ac9-3e15-4c77-8630-225b01cc20ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:17.865693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1652ac9-3e15-4c77-8630-225b01cc20ea None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:17.876492 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 641/1270] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:17.882033 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa383327-eee7-43c0-9c0d-ba251c973084 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:17.913835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:17.991789 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa383327-eee7-43c0-9c0d-ba251c973084 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:18.027028 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc21f8-eb75-43be-b2df-57ef68d5394d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:18.027830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc21f8-eb75-43be-b2df-57ef68d5394d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:18.028096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc21f8-eb75-43be-b2df-57ef68d5394d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:18.028370 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cc21f8-eb75-43be-b2df-57ef68d5394d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:18.034812 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc21f8-eb75-43be-b2df-57ef68d5394d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:18.036225 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cc21f8-eb75-43be-b2df-57ef68d5394d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:18.049781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-80cc21f8-eb75-43be-b2df-57ef68d5394d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:51:18.049781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:18.049781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:18.049781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:18.049781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:18.049781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:18.049781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:18.049781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:18.049781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:18.049781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:18.049781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:18.049781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:18.049781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:51:18.049781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:51:18.049781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:51:18.050893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:51:18.050893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:18.050893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:18.050893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:18.050893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:18.050893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:18.050893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:18.050893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:18.050893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:18.050893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:18.050893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:18.050893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:18.050893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:18.050893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:18.050893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:18.050893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:18.050893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:18.051980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:51:18.051980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:51:18.051980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:51:18.051980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:51:18.051980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:51:18.051980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:51:18.051980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:51:18.051980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:51:18.051980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 630/1271] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:51:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:51:18.056750 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1652ac9-3e15-4c77-8630-225b01cc20ea None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:18.113505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:18.154709 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1652ac9-3e15-4c77-8630-225b01cc20ea None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:18.186848 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63531fd7-a568-4d7f-b821-5c4681148292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:18.187436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63531fd7-a568-4d7f-b821-5c4681148292 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:18.187721 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63531fd7-a568-4d7f-b821-5c4681148292 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:18.187999 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63531fd7-a568-4d7f-b821-5c4681148292 None admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:18.188771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63531fd7-a568-4d7f-b821-5c4681148292 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:18.189083 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63531fd7-a568-4d7f-b821-5c4681148292 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:18.190496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63531fd7-a568-4d7f-b821-5c4681148292 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:18.202656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 642/1272] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:51:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:18.208575 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc21f8-eb75-43be-b2df-57ef68d5394d None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:18.299399 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:18.308922 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cc21f8-eb75-43be-b2df-57ef68d5394d None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:18.341974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cba577a-9052-4717-84ca-60daae8aec6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:18.342884 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba577a-9052-4717-84ca-60daae8aec6b None admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:18.343172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba577a-9052-4717-84ca-60daae8aec6b None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:18.343449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cba577a-9052-4717-84ca-60daae8aec6b None admin] PATH_INFO: `/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/f9593939641c4c7797c1981206b40e7d/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:18.345381 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba577a-9052-4717-84ca-60daae8aec6b None admin] RBAC: Authorizing `identity:create_grant(project_id=bc48914270df4db996bd2a4b5b9fc777, user_id=f9593939641c4c7797c1981206b40e7d, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:18.346851 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cba577a-9052-4717-84ca-60daae8aec6b None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:18.358560 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 631/1273] 10.222.0.17 () {58 vars in 1439 bytes} [Thu Jun 16 22:51:18 2022] PUT /identity/v3/projects/bc48914270df4db996bd2a4b5b9fc777/users/f9593939641c4c7797c1981206b40e7d/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:51:20.439049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c2f26-174e-45ea-8c77-e085418df09d None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:20.439566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c2f26-174e-45ea-8c77-e085418df09d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:20.439654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702c2f26-174e-45ea-8c77-e085418df09d None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:20.440959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 643/1274] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:51:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:51:20.446066 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2069e3-14be-4af7-b476-9ee41a5061f1 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:20.446506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2069e3-14be-4af7-b476-9ee41a5061f1 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:20.446652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2069e3-14be-4af7-b476-9ee41a5061f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:20.463491 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-bb2069e3-14be-4af7-b476-9ee41a5061f1 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:51:20.464675 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-bb2069e3-14be-4af7-b476-9ee41a5061f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:51:20.529489 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 632/1275] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:51:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:51:20.562126 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c2f26-174e-45ea-8c77-e085418df09d None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:20.660645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702c2f26-174e-45ea-8c77-e085418df09d None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:20.677665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:20.694546 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2716e5d-98f0-4269-be23-5f5ccaa6c68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:20.695359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2716e5d-98f0-4269-be23-5f5ccaa6c68f None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:20.695645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2716e5d-98f0-4269-be23-5f5ccaa6c68f None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:20.695948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2716e5d-98f0-4269-be23-5f5ccaa6c68f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:20.705789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2716e5d-98f0-4269-be23-5f5ccaa6c68f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 22:51:20.713435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-a2716e5d-98f0-4269-be23-5f5ccaa6c68f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:51:20.713435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:20.713435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:20.713435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:20.713435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:20.713435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:20.713435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:20.713435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:20.713435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:20.713435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:20.713435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:20.713435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:20.713435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 22:51:20.713435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 22:51:20.713435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 22:51:20.713435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 22:51:20.714513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 22:51:20.714513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 22:51:20.714513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 22:51:20.714513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 22:51:20.714513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:20.714513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:20.714513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:20.714513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:20.714513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:20.714513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:20.714513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:20.714513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:20.714513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:20.714513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:20.714513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:20.714513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:20.715558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:20.715558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:20.715558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:20.715558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:20.715558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 22:51:20.715558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 22:51:20.715558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 22:51:20.715558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 22:51:20.715558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 22:51:20.715558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 22:51:20.715558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 22:51:20.715558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:20.715558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 644/1276] 10.222.0.17 () {56 vars in 1096 bytes} [Thu Jun 16 22:51:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:20.720410 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2069e3-14be-4af7-b476-9ee41a5061f1 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:20.798525 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:20.818993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2069e3-14be-4af7-b476-9ee41a5061f1 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:20.851229 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a571c-4820-49a2-883e-ebd9183814ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:20.851890 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a571c-4820-49a2-883e-ebd9183814ee None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:20.852176 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a571c-4820-49a2-883e-ebd9183814ee None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:20.852459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42a571c-4820-49a2-883e-ebd9183814ee None admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:20.853190 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a571c-4820-49a2-883e-ebd9183814ee None admin] RBAC: Adding query filter params (name=admin) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:20.853496 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a571c-4820-49a2-883e-ebd9183814ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:20.855451 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42a571c-4820-49a2-883e-ebd9183814ee None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:20.863161 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 633/1277] 10.222.0.17 () {56 vars in 1099 bytes} [Thu Jun 16 22:51:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 22:51:20.869050 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2716e5d-98f0-4269-be23-5f5ccaa6c68f None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:20.965194 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2716e5d-98f0-4269-be23-5f5ccaa6c68f None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:20.999873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7d6d8-191b-455f-956d-5134617115f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:21.000733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7d6d8-191b-455f-956d-5134617115f5 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:21.001065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7d6d8-191b-455f-956d-5134617115f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:21.001383 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7d6d8-191b-455f-956d-5134617115f5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:21.012523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7d6d8-191b-455f-956d-5134617115f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:21.013967 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7d6d8-191b-455f-956d-5134617115f5 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:21.035209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-3cd7d6d8-191b-455f-956d-5134617115f5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:21.035209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:21.035209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:21.035209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:21.035209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:21.035209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:21.035209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:21.035209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:21.035209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:21.035209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:21.035209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:21.035209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:21.035209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:51:21.035209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:51:21.035209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:51:21.036545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:51:21.036545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:21.036545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:21.036545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:21.036545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:21.036545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:21.036545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:21.036545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:21.036545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:21.036545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:21.036545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:21.036545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:21.036545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:21.036545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:21.036545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:21.036545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:21.036545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:21.037913 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:51:21.037913 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:51:21.037913 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:21.037913 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:21.042250 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 645/1278] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:51:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:21.043443 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a571c-4820-49a2-883e-ebd9183814ee None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:21.138351 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42a571c-4820-49a2-883e-ebd9183814ee None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:21.170763 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351bd16-f98a-4d5a-928f-1ecbcd45be96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:21.171402 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351bd16-f98a-4d5a-928f-1ecbcd45be96 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:21.171688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351bd16-f98a-4d5a-928f-1ecbcd45be96 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:21.171967 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4351bd16-f98a-4d5a-928f-1ecbcd45be96 None admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:21.172696 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351bd16-f98a-4d5a-928f-1ecbcd45be96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:21.173004 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351bd16-f98a-4d5a-928f-1ecbcd45be96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:21.174347 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4351bd16-f98a-4d5a-928f-1ecbcd45be96 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:21.185552 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 634/1279] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:21.191123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7d6d8-191b-455f-956d-5134617115f5 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:21.221710 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:21.289917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7d6d8-191b-455f-956d-5134617115f5 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:21.322256 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412fb0b-0632-45db-8522-6759585514d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:21.323102 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412fb0b-0632-45db-8522-6759585514d8 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:21.323386 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412fb0b-0632-45db-8522-6759585514d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:21.323660 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6412fb0b-0632-45db-8522-6759585514d8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:21.331607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412fb0b-0632-45db-8522-6759585514d8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:21.333016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6412fb0b-0632-45db-8522-6759585514d8 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:21.347375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-6412fb0b-0632-45db-8522-6759585514d8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 22:51:21.347375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:21.347375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:21.347375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:21.347375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:21.347375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:21.347375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:21.347375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:21.347375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:21.347375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:21.347375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:21.347375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:21.347375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 22:51:21.347375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 22:51:21.347375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 22:51:21.347375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 22:51:21.348481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:21.348481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:21.348481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 22:51:21.348481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:51:21.348481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 22:51:21.348481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 22:51:21.348481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:21.348481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:21.348481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:21.348481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:21.348481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:21.348481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:21.348481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:21.348481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:21.348481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:21.348481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:21.348481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:21.348481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:21.349564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:21.349564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:21.349564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 22:51:21.349564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 22:51:21.349564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 22:51:21.349564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 22:51:21.349564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 22:51:21.349564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 22:51:21.349564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 22:51:21.349564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:21.349564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 646/1280] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:51:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:21.354948 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351bd16-f98a-4d5a-928f-1ecbcd45be96 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:21.358630 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:21.451290 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4351bd16-f98a-4d5a-928f-1ecbcd45be96 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:21.484377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c8f5d-1831-498c-b93f-e2e6e03ca3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:21.485034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c8f5d-1831-498c-b93f-e2e6e03ca3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:21.485317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c8f5d-1831-498c-b93f-e2e6e03ca3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:21.485590 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87c8f5d-1831-498c-b93f-e2e6e03ca3f4 None admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:21.486429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c8f5d-1831-498c-b93f-e2e6e03ca3f4 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:21.486770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c8f5d-1831-498c-b93f-e2e6e03ca3f4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:21.488217 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87c8f5d-1831-498c-b93f-e2e6e03ca3f4 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:21.504385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 635/1281] 10.222.0.17 () {56 vars in 1143 bytes} [Thu Jun 16 22:51:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:21.509889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412fb0b-0632-45db-8522-6759585514d8 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:21.597854 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:21.607629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6412fb0b-0632-45db-8522-6759585514d8 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:21.639865 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4db148-1419-4ed7-93b6-7dfa7c650dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:21.640498 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4db148-1419-4ed7-93b6-7dfa7c650dcd None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:21.640784 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4db148-1419-4ed7-93b6-7dfa7c650dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:21.641062 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4db148-1419-4ed7-93b6-7dfa7c650dcd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:21.705307 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4db148-1419-4ed7-93b6-7dfa7c650dcd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:21.706700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4db148-1419-4ed7-93b6-7dfa7c650dcd None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:21.790910 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:21.801477 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 647/1282] 10.222.0.17 () {58 vars in 1282 bytes} [Thu Jun 16 22:51:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 22:51:21.807786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c8f5d-1831-498c-b93f-e2e6e03ca3f4 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:21.870182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:21.907919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87c8f5d-1831-498c-b93f-e2e6e03ca3f4 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:21.940211 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f46d8-2a5f-4b73-822b-b354b1bf233b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:21.940993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f46d8-2a5f-4b73-822b-b354b1bf233b None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:21.941274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f46d8-2a5f-4b73-822b-b354b1bf233b None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:21.941547 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f46d8-2a5f-4b73-822b-b354b1bf233b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:21.952320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f46d8-2a5f-4b73-822b-b354b1bf233b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:21.953894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f46d8-2a5f-4b73-822b-b354b1bf233b None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:21.970624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-469f46d8-2a5f-4b73-822b-b354b1bf233b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:21.970624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:21.970624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:21.970624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:21.970624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:21.970624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:21.970624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:21.970624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:21.970624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:21.970624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:21.970624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:21.970624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:21.970624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 22:51:21.970624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 22:51:21.970624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 22:51:21.971611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 22:51:21.971611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:21.971611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:21.971611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:21.971611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:21.971611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:21.971611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:21.971611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:21.971611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:21.971611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:21.971611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:21.971611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:21.971611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:21.971611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:21.971611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:21.971611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:21.971611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:21.972694 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 22:51:21.972694 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 22:51:21.972694 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 22:51:21.972694 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:51:21.972694 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 636/1283] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:51:21.977465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4db148-1419-4ed7-93b6-7dfa7c650dcd None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:22.074280 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4db148-1419-4ed7-93b6-7dfa7c650dcd None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:22.106220 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ceedd-b687-4104-9020-94dfae8ba41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:22.106894 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ceedd-b687-4104-9020-94dfae8ba41b None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:22.107178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ceedd-b687-4104-9020-94dfae8ba41b None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:22.107451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63ceedd-b687-4104-9020-94dfae8ba41b None admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:22.108154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ceedd-b687-4104-9020-94dfae8ba41b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:22.108457 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ceedd-b687-4104-9020-94dfae8ba41b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:22.109794 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63ceedd-b687-4104-9020-94dfae8ba41b None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:22.121021 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 648/1284] 10.222.0.17 () {56 vars in 1109 bytes} [Thu Jun 16 22:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:22.126543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f46d8-2a5f-4b73-822b-b354b1bf233b None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:22.229213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f46d8-2a5f-4b73-822b-b354b1bf233b None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:22.242888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:22.263096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba953472-4367-4eb4-9949-c03c9e906082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:22.263888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba953472-4367-4eb4-9949-c03c9e906082 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:22.264176 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba953472-4367-4eb4-9949-c03c9e906082 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:22.264457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba953472-4367-4eb4-9949-c03c9e906082 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:22.270932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba953472-4367-4eb4-9949-c03c9e906082 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:22.272367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba953472-4367-4eb4-9949-c03c9e906082 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:22.285954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application [None req-ba953472-4367-4eb4-9949-c03c9e906082 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:51:22.285954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:22.285954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:22.285954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:22.285954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:22.285954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:22.285954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:22.285954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:22.285954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:22.285954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:22.285954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:22.285954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:22.285954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 22:51:22.285954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 22:51:22.285954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 22:51:22.286990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 22:51:22.286990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:22.286990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:22.286990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:22.286990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:22.286990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:22.286990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:22.286990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:22.286990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:22.286990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:22.286990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:22.286990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:22.286990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:22.286990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:22.286990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:22.286990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:22.286990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:22.293149 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 22:51:22.293149 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 22:51:22.293149 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 22:51:22.293149 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 22:51:22.293149 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 22:51:22.293149 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 22:51:22.293149 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 22:51:22.293149 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: ERROR keystone.server.flask.application Jun 16 22:51:22.293149 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 637/1285] 10.222.0.17 () {56 vars in 1111 bytes} [Thu Jun 16 22:51:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:22.293758 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ceedd-b687-4104-9020-94dfae8ba41b None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:22.345541 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:22.394790 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63ceedd-b687-4104-9020-94dfae8ba41b None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:22.427359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676f78cc-58ce-40d7-8544-53c3335684a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:22.428013 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f78cc-58ce-40d7-8544-53c3335684a6 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:22.428300 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f78cc-58ce-40d7-8544-53c3335684a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:22.428579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676f78cc-58ce-40d7-8544-53c3335684a6 None admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:22.429358 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f78cc-58ce-40d7-8544-53c3335684a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:22.429673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f78cc-58ce-40d7-8544-53c3335684a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:22.431133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-676f78cc-58ce-40d7-8544-53c3335684a6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:22.442656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 649/1286] 10.222.0.17 () {56 vars in 1148 bytes} [Thu Jun 16 22:51:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:22.449162 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba953472-4367-4eb4-9949-c03c9e906082 None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:22.544782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba953472-4367-4eb4-9949-c03c9e906082 None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:22.576447 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f181b883-3dd6-4876-b378-2b10128b80cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:22.577079 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181b883-3dd6-4876-b378-2b10128b80cb None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:22.577360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181b883-3dd6-4876-b378-2b10128b80cb None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:22.577635 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f181b883-3dd6-4876-b378-2b10128b80cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:22.578443 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181b883-3dd6-4876-b378-2b10128b80cb None admin] RBAC: Adding query filter params (role.id=bca7a25be0b4426e85d561e14e1bf13e, scope.project.id=bc48914270df4db996bd2a4b5b9fc777, user.id=f9593939641c4c7797c1981206b40e7d) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:22.578783 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181b883-3dd6-4876-b378-2b10128b80cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:22.580189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f181b883-3dd6-4876-b378-2b10128b80cb None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:22.585766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 638/1287] 10.222.0.17 () {56 vars in 1374 bytes} [Thu Jun 16 22:51:22 2022] GET /identity/v3/role_assignments?user.id=f9593939641c4c7797c1981206b40e7d&scope.project.id=bc48914270df4db996bd2a4b5b9fc777&role.id=bca7a25be0b4426e85d561e14e1bf13e => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:24.682669 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a4690-1627-48ef-9ae4-dfb29f7e576e None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:24.683213 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a4690-1627-48ef-9ae4-dfb29f7e576e None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:24.683213 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23a4690-1627-48ef-9ae4-dfb29f7e576e None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:24.684581 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 650/1288] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:51:24.689652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e12a0-c4c4-42f7-9026-2879bc81653c None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:24.690085 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e12a0-c4c4-42f7-9026-2879bc81653c None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:24.690222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90e12a0-c4c4-42f7-9026-2879bc81653c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:24.707284 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-b90e12a0-c4c4-42f7-9026-2879bc81653c None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:51:24.708472 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-b90e12a0-c4c4-42f7-9026-2879bc81653c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:51:24.737296 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:24.774321 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 639/1289] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:51:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:51:24.796595 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a4690-1627-48ef-9ae4-dfb29f7e576e None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:24.898358 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23a4690-1627-48ef-9ae4-dfb29f7e576e None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:24.909480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:24.933291 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e7bf9-7015-4ef1-98ef-d86507223b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:24.934019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e7bf9-7015-4ef1-98ef-d86507223b31 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:24.934304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e7bf9-7015-4ef1-98ef-d86507223b31 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:24.934578 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20e7bf9-7015-4ef1-98ef-d86507223b31 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:24.935252 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e7bf9-7015-4ef1-98ef-d86507223b31 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:24.936595 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20e7bf9-7015-4ef1-98ef-d86507223b31 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:24.948656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-d20e7bf9-7015-4ef1-98ef-d86507223b31 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 22:51:24.948656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:24.948656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:51:24.948656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:51:24.948656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:51:24.948656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:51:24.948656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:51:24.948656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:51:24.948656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 22:51:24.948656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:24.948656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:51:24.948656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:24.948656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:24.948656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:24.948656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:24.948656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:24.948656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:24.949881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:24.949881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:24.949881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:24.949881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:24.949881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:24.949881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:24.949881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:51:24.949881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:51:24.949881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:51:24.949881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:51:24.949881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:24.949881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:24.949881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:24.949881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:24.949881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:24.949881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:24.951014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:24.951014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:24.951014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:24.951014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:24.951014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:24.951014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:24.951014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:24.951014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:24.951014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:24.951014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:24.951014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:51:24.951014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:51:24.951014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 22:51:24.951014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:24.951014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 651/1290] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 22:51:24 2022] GET /identity/v3/services/placement => generated 90 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:24.955875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e12a0-c4c4-42f7-9026-2879bc81653c None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:25.059622 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90e12a0-c4c4-42f7-9026-2879bc81653c None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:25.094008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd9e864-470b-49fb-91d6-34c8cfbd384d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:25.094690 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9e864-470b-49fb-91d6-34c8cfbd384d None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:25.094977 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9e864-470b-49fb-91d6-34c8cfbd384d None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:25.095257 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd9e864-470b-49fb-91d6-34c8cfbd384d None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:25.095962 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9e864-470b-49fb-91d6-34c8cfbd384d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:25.096271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9e864-470b-49fb-91d6-34c8cfbd384d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:25.097631 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd9e864-470b-49fb-91d6-34c8cfbd384d None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:25.102356 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 640/1291] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jun 16 22:51:24 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:25.108266 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e7bf9-7015-4ef1-98ef-d86507223b31 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:25.212570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20e7bf9-7015-4ef1-98ef-d86507223b31 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:25.245528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f13861fa-275b-45ff-8dc2-e7632e3441ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:25.246180 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13861fa-275b-45ff-8dc2-e7632e3441ca None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:25.246464 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13861fa-275b-45ff-8dc2-e7632e3441ca None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:25.246782 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f13861fa-275b-45ff-8dc2-e7632e3441ca None admin] PATH_INFO: `/v3/services` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:25.247482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13861fa-275b-45ff-8dc2-e7632e3441ca None admin] RBAC: Adding query filter params (type=placement) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:25.247791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13861fa-275b-45ff-8dc2-e7632e3441ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:25.249146 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f13861fa-275b-45ff-8dc2-e7632e3441ca None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:25.253487 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 652/1292] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jun 16 22:51:25 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:27.315814 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3436d7-d3e1-4cf7-b38e-1d0664131f1b None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:27.316354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3436d7-d3e1-4cf7-b38e-1d0664131f1b None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:27.316354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3436d7-d3e1-4cf7-b38e-1d0664131f1b None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:27.317722 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 641/1293] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:51:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:51:27.323194 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bcf30-2963-453a-8056-7ee7363b164c None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:27.323607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bcf30-2963-453a-8056-7ee7363b164c None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:27.323755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169bcf30-2963-453a-8056-7ee7363b164c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:27.340126 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-169bcf30-2963-453a-8056-7ee7363b164c None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:51:27.341303 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-169bcf30-2963-453a-8056-7ee7363b164c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:51:27.369882 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:27.406674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 653/1294] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:51:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:51:27.429181 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3436d7-d3e1-4cf7-b38e-1d0664131f1b None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:27.459506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:27.528780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3436d7-d3e1-4cf7-b38e-1d0664131f1b None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:27.561335 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72877147-cbc4-4c24-8ee4-9e6e77b84f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:27.561981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72877147-cbc4-4c24-8ee4-9e6e77b84f90 None admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:27.562264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72877147-cbc4-4c24-8ee4-9e6e77b84f90 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:27.562540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72877147-cbc4-4c24-8ee4-9e6e77b84f90 None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:27.563272 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72877147-cbc4-4c24-8ee4-9e6e77b84f90 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:27.564628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72877147-cbc4-4c24-8ee4-9e6e77b84f90 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:27.577046 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 642/1295] 10.222.0.17 () {60 vars in 1141 bytes} [Thu Jun 16 22:51:27 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:51:29.655607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a906f10-6f06-4e0e-8f9d-e033eda4ae48 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:29.656161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a906f10-6f06-4e0e-8f9d-e033eda4ae48 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:29.656161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a906f10-6f06-4e0e-8f9d-e033eda4ae48 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:29.657514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 654/1296] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:51:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:51:29.662723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464e505-f5cb-4972-8d52-68c4966591f6 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:29.663154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464e505-f5cb-4972-8d52-68c4966591f6 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:29.663292 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d464e505-f5cb-4972-8d52-68c4966591f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:29.680168 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-d464e505-f5cb-4972-8d52-68c4966591f6 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:51:29.681354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d464e505-f5cb-4972-8d52-68c4966591f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:51:29.751688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 643/1297] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:51:29 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:51:29.773455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a906f10-6f06-4e0e-8f9d-e033eda4ae48 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:29.871147 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a906f10-6f06-4e0e-8f9d-e033eda4ae48 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:29.903706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b7b5c-6619-4525-b7e5-731ddecf13e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:29.904489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b7b5c-6619-4525-b7e5-731ddecf13e6 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:29.904775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b7b5c-6619-4525-b7e5-731ddecf13e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:29.905051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2b7b5c-6619-4525-b7e5-731ddecf13e6 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:29.905737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b7b5c-6619-4525-b7e5-731ddecf13e6 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:29.907069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2b7b5c-6619-4525-b7e5-731ddecf13e6 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:29.920813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-4c2b7b5c-6619-4525-b7e5-731ddecf13e6 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 22:51:29.920813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:29.920813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:51:29.920813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:51:29.920813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:51:29.920813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:51:29.920813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:51:29.920813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:51:29.920813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 22:51:29.920813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:29.920813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:51:29.920813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:29.920813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:29.920813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:29.920813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:29.920813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:29.920813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:29.921827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:29.921827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:29.921827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:29.921827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:29.921827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:29.921827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:29.921827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:51:29.921827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:51:29.921827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:51:29.921827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:51:29.921827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:29.921827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:29.921827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:29.921827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:29.921827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:29.921827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:29.922725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:29.922725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:29.922725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:29.922725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:29.922725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:29.922725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:29.922725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:29.922725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:29.922725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:29.922725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:29.922725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:51:29.922725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:51:29.922725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 22:51:29.922725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:29.922725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 655/1298] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 22:51:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 22:51:29.927986 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464e505-f5cb-4972-8d52-68c4966591f6 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:29.946251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:30.026835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d464e505-f5cb-4972-8d52-68c4966591f6 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:30.059287 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0e147-d9eb-4c0d-b7f3-e6c16f83b91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:30.059947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0e147-d9eb-4c0d-b7f3-e6c16f83b91f None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:30.060236 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0e147-d9eb-4c0d-b7f3-e6c16f83b91f None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:30.060518 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b0e147-d9eb-4c0d-b7f3-e6c16f83b91f None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:30.061219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0e147-d9eb-4c0d-b7f3-e6c16f83b91f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:30.061530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0e147-d9eb-4c0d-b7f3-e6c16f83b91f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:30.062920 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b0e147-d9eb-4c0d-b7f3-e6c16f83b91f None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:30.067929 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 644/1299] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jun 16 22:51:29 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:30.073886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b7b5c-6619-4525-b7e5-731ddecf13e6 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:30.104881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:30.171241 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2b7b5c-6619-4525-b7e5-731ddecf13e6 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:30.202815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393e3fd3-3c0c-4a23-a46e-c600bd0d6cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:30.203462 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e3fd3-3c0c-4a23-a46e-c600bd0d6cde None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:30.203751 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e3fd3-3c0c-4a23-a46e-c600bd0d6cde None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:30.204028 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393e3fd3-3c0c-4a23-a46e-c600bd0d6cde None admin] PATH_INFO: `/v3/endpoints` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:30.204805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e3fd3-3c0c-4a23-a46e-c600bd0d6cde None admin] RBAC: Adding query filter params (interface=public, service_id=b06a3ca5c6a94cb887f2c873f868e974, region_id=RegionOne) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:30.205120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e3fd3-3c0c-4a23-a46e-c600bd0d6cde None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:30.206508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393e3fd3-3c0c-4a23-a46e-c600bd0d6cde None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:30.211644 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 656/1300] 10.222.0.17 () {56 vars in 1251 bytes} [Thu Jun 16 22:51:30 2022] GET /identity/v3/endpoints?service_id=b06a3ca5c6a94cb887f2c873f868e974&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:30.217658 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0e147-d9eb-4c0d-b7f3-e6c16f83b91f None admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:30.308210 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:30.326040 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b0e147-d9eb-4c0d-b7f3-e6c16f83b91f None admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:30.359161 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a810a6a-e09b-4be9-a486-12fe8a6f5f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:30.359814 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a810a6a-e09b-4be9-a486-12fe8a6f5f9a None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:30.360098 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a810a6a-e09b-4be9-a486-12fe8a6f5f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:30.360374 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a810a6a-e09b-4be9-a486-12fe8a6f5f9a None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:30.361043 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a810a6a-e09b-4be9-a486-12fe8a6f5f9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:30.362394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a810a6a-e09b-4be9-a486-12fe8a6f5f9a None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:30.369090 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 645/1301] 10.222.0.17 () {56 vars in 1087 bytes} [Thu Jun 16 22:51:30 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 22:51:32.468280 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d41c2-3f4b-4f84-bb0e-790e3bf280e8 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:32.468816 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d41c2-3f4b-4f84-bb0e-790e3bf280e8 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:32.468816 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d41c2-3f4b-4f84-bb0e-790e3bf280e8 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:32.470215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 657/1302] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:51:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:51:32.475165 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f99ad-7d1e-4458-8afa-896c31634a52 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:32.475543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f99ad-7d1e-4458-8afa-896c31634a52 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:32.475725 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f99ad-7d1e-4458-8afa-896c31634a52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:32.492477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-626f99ad-7d1e-4458-8afa-896c31634a52 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:51:32.493584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-626f99ad-7d1e-4458-8afa-896c31634a52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:51:32.557778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 646/1303] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:51:32 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 22:51:32.579617 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d41c2-3f4b-4f84-bb0e-790e3bf280e8 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:32.672498 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:32.679486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277d41c2-3f4b-4f84-bb0e-790e3bf280e8 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:32.711568 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e37849-6ba2-4bcd-9364-482fd9d92170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:32.712351 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e37849-6ba2-4bcd-9364-482fd9d92170 None admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:32.712637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e37849-6ba2-4bcd-9364-482fd9d92170 None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:32.712944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e37849-6ba2-4bcd-9364-482fd9d92170 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:32.713590 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e37849-6ba2-4bcd-9364-482fd9d92170 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:32.714969 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e37849-6ba2-4bcd-9364-482fd9d92170 None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:32.726523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application [None req-a2e37849-6ba2-4bcd-9364-482fd9d92170 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 22:51:32.726523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:32.726523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 22:51:32.726523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 22:51:32.726523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 22:51:32.726523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 22:51:32.726523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 22:51:32.726523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:51:32.726523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 22:51:32.726523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:32.726523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 22:51:32.726523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:32.726523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 22:51:32.726523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 22:51:32.726523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 22:51:32.726523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 22:51:32.726523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 22:51:32.727687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 22:51:32.727687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 22:51:32.727687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 22:51:32.727687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 22:51:32.727687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 22:51:32.727687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 22:51:32.727687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 22:51:32.727687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 22:51:32.727687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 22:51:32.727687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 22:51:32.727687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 22:51:32.727687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 22:51:32.727687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 22:51:32.727687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 22:51:32.727687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 22:51:32.727687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 22:51:32.729349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 22:51:32.729349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application with Lock( Jun 16 22:51:32.729349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 22:51:32.729349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self._enter() Jun 16 22:51:32.729349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 22:51:32.729349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 22:51:32.729349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 22:51:32.729349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application return self.creator() Jun 16 22:51:32.729349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 22:51:32.729349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application created_value = creator( Jun 16 22:51:32.729349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 22:51:32.729349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 22:51:32.729349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 22:51:32.729349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: ERROR keystone.server.flask.application Jun 16 22:51:32.729349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 658/1304] 10.222.0.17 () {56 vars in 1117 bytes} [Thu Jun 16 22:51:32 2022] GET /identity/v3/services/placement => generated 90 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 22:51:32.734080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f99ad-7d1e-4458-8afa-896c31634a52 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:32.806158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:32.828767 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f99ad-7d1e-4458-8afa-896c31634a52 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:32.861651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21287af4-177f-41c8-a859-0d2e54709c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:32.862301 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21287af4-177f-41c8-a859-0d2e54709c19 None admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:32.862583 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21287af4-177f-41c8-a859-0d2e54709c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:32.862884 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21287af4-177f-41c8-a859-0d2e54709c19 None admin] PATH_INFO: `/v3/services` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:32.863575 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21287af4-177f-41c8-a859-0d2e54709c19 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:51:32.863878 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21287af4-177f-41c8-a859-0d2e54709c19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:32.865216 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21287af4-177f-41c8-a859-0d2e54709c19 None admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:32.870812 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 647/1305] 10.222.0.17 () {56 vars in 1116 bytes} [Thu Jun 16 22:51:32 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:51:32.876402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e37849-6ba2-4bcd-9364-482fd9d92170 None admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:51:32.975760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e37849-6ba2-4bcd-9364-482fd9d92170 None admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:51:33.008613 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e7d712-2a76-4a94-b375-2aa379b70e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:51:33.009267 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7d712-2a76-4a94-b375-2aa379b70e1f None admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:51:33.009574 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7d712-2a76-4a94-b375-2aa379b70e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:51:33.009853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e7d712-2a76-4a94-b375-2aa379b70e1f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:51:33.010617 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e7d712-2a76-4a94-b375-2aa379b70e1f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:51:33.011980 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e7d712-2a76-4a94-b375-2aa379b70e1f None admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:51:33.030072 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 659/1306] 10.222.0.17 () {60 vars in 1144 bytes} [Thu Jun 16 22:51:32 2022] POST /identity/v3/endpoints => generated 331 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:51:42.867228 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=30 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:51:43.025940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:52:35.472487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2483c-7647-4c0e-83c7-bd8c53661820 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:35.473078 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2483c-7647-4c0e-83c7-bd8c53661820 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:35.473078 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f2483c-7647-4c0e-83c7-bd8c53661820 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:35.478899 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 648/1307] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:52:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:52:35.480259 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68232fa-74b0-442d-a01c-a98dd0f7fca9 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:35.480677 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68232fa-74b0-442d-a01c-a98dd0f7fca9 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:35.480814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68232fa-74b0-442d-a01c-a98dd0f7fca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:35.681404 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-b68232fa-74b0-442d-a01c-a98dd0f7fca9 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:52:35.686195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b68232fa-74b0-442d-a01c-a98dd0f7fca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:35.895333 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 660/1308] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:52:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:52:35.933790 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590e448-4ff8-48d3-a211-182ea78447bf None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:35.934081 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590e448-4ff8-48d3-a211-182ea78447bf None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:35.934558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0590e448-4ff8-48d3-a211-182ea78447bf None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:35.936772 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 649/1309] 10.222.0.17 () {54 vars in 964 bytes} [Thu Jun 16 22:52:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:52:35.943170 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d2a4f-96c7-4d33-8dba-438ba318fa14 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:35.944033 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d2a4f-96c7-4d33-8dba-438ba318fa14 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:35.944317 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5d2a4f-96c7-4d33-8dba-438ba318fa14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:36.023979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-2b5d2a4f-96c7-4d33-8dba-438ba318fa14 None None] MFA Rules not processed for user `6b794f7aa2a7488e91f7d9ea35a45d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:52:36.025195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-2b5d2a4f-96c7-4d33-8dba-438ba318fa14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:36.048079 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:52:36.112054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 661/1310] 10.222.0.17 () {58 vars in 1061 bytes} [Thu Jun 16 22:52:35 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:52:36.122036 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590e448-4ff8-48d3-a211-182ea78447bf None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:52:36.124080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-0590e448-4ff8-48d3-a211-182ea78447bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:36.263199 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0590e448-4ff8-48d3-a211-182ea78447bf None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:52:36.358324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb49b35-af25-4617-8893-2930216c5d2f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:52:36.359179 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb49b35-af25-4617-8893-2930216c5d2f service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:36.359474 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb49b35-af25-4617-8893-2930216c5d2f service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:36.359755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb49b35-af25-4617-8893-2930216c5d2f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:36.361482 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-0bb49b35-af25-4617-8893-2930216c5d2f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:36.424698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:52:36.486936 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb49b35-af25-4617-8893-2930216c5d2f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:52:36.488490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb49b35-af25-4617-8893-2930216c5d2f service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:52:36.509191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 650/1311] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:52:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:52:38.960236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d7e3e-32da-4f40-930b-2ba42c63a526 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:38.960810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d7e3e-32da-4f40-930b-2ba42c63a526 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:38.960810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9d7e3e-32da-4f40-930b-2ba42c63a526 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:38.962172 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 662/1312] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:52:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:52:38.967378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cf022-dd77-4474-ae37-fbd037a087c5 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:38.967798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cf022-dd77-4474-ae37-fbd037a087c5 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:38.967972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7cf022-dd77-4474-ae37-fbd037a087c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:39.039667 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-cb7cf022-dd77-4474-ae37-fbd037a087c5 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:52:39.040917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-cb7cf022-dd77-4474-ae37-fbd037a087c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:39.136229 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 651/1313] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:52:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:52:39.177711 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d7e3e-32da-4f40-930b-2ba42c63a526 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:52:39.179279 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-3e9d7e3e-32da-4f40-930b-2ba42c63a526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:39.283072 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9d7e3e-32da-4f40-930b-2ba42c63a526 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:52:39.356829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:52:39.359654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f34212-e5d7-4103-9590-79df2bf6074d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:52:39.360399 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f34212-e5d7-4103-9590-79df2bf6074d service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:39.360932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f34212-e5d7-4103-9590-79df2bf6074d service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:39.361470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f34212-e5d7-4103-9590-79df2bf6074d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:39.363437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-56f34212-e5d7-4103-9590-79df2bf6074d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:39.485705 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f34212-e5d7-4103-9590-79df2bf6074d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:52:39.487123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f34212-e5d7-4103-9590-79df2bf6074d service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:52:39.507248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 663/1314] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:52:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:52:41.898013 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3f6be-245e-4f0e-bdc0-7002373fbc91 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:41.898462 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3f6be-245e-4f0e-bdc0-7002373fbc91 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:41.898637 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a3f6be-245e-4f0e-bdc0-7002373fbc91 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:41.899973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 652/1315] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:52:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:52:41.905044 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa17f9-a5f5-4e25-ad7d-a52ade847736 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:41.905443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa17f9-a5f5-4e25-ad7d-a52ade847736 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:41.905610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffa17f9-a5f5-4e25-ad7d-a52ade847736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:41.969032 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-fffa17f9-a5f5-4e25-ad7d-a52ade847736 None None] MFA Rules not processed for user `506c986a39184b05b22b043ea6c2f174`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:52:41.970231 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-fffa17f9-a5f5-4e25-ad7d-a52ade847736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:42.022712 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:52:42.066439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 664/1316] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:52:41 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:52:42.105311 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3f6be-245e-4f0e-bdc0-7002373fbc91 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:52:42.106861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c2a3f6be-245e-4f0e-bdc0-7002373fbc91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:42.116923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:52:42.207953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a3f6be-245e-4f0e-bdc0-7002373fbc91 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:52:42.279560 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607f8df-13f2-4981-94de-aa4386145935 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:52:42.280201 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607f8df-13f2-4981-94de-aa4386145935 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:42.280483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607f8df-13f2-4981-94de-aa4386145935 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:42.280757 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607f8df-13f2-4981-94de-aa4386145935 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:42.282368 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-a607f8df-13f2-4981-94de-aa4386145935 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:42.342400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:52:42.389432 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607f8df-13f2-4981-94de-aa4386145935 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:52:42.390912 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607f8df-13f2-4981-94de-aa4386145935 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:52:42.408588 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 653/1317] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:52:42 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:52:45.036396 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7400d0-0382-40b6-8627-20ea6eae47f1 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:45.037494 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7400d0-0382-40b6-8627-20ea6eae47f1 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:45.037494 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7400d0-0382-40b6-8627-20ea6eae47f1 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:45.038910 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 665/1318] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:52:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:52:45.043945 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667899c-7476-4a1f-9e1f-794865040169 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:45.044377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667899c-7476-4a1f-9e1f-794865040169 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:45.044515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667899c-7476-4a1f-9e1f-794865040169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:45.107815 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-8667899c-7476-4a1f-9e1f-794865040169 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:52:45.109022 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-8667899c-7476-4a1f-9e1f-794865040169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:45.202692 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 654/1319] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:52:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:52:45.241375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7400d0-0382-40b6-8627-20ea6eae47f1 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:52:45.242942 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-1a7400d0-0382-40b6-8627-20ea6eae47f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:45.345856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7400d0-0382-40b6-8627-20ea6eae47f1 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:52:45.417031 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb4edd4-a80a-4ca4-a198-8b089b00a55e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:52:45.417684 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4edd4-a80a-4ca4-a198-8b089b00a55e service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:45.417972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4edd4-a80a-4ca4-a198-8b089b00a55e service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:45.418252 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb4edd4-a80a-4ca4-a198-8b089b00a55e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:45.419945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ddb4edd4-a80a-4ca4-a198-8b089b00a55e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:45.427439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:52:45.527865 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4edd4-a80a-4ca4-a198-8b089b00a55e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:52:45.529277 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb4edd4-a80a-4ca4-a198-8b089b00a55e service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:52:45.546657 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 666/1320] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:52:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:52:48.337775 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754681d5-b1d8-48fa-ae87-c3d5a824f42e None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:48.338295 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754681d5-b1d8-48fa-ae87-c3d5a824f42e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:48.338789 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754681d5-b1d8-48fa-ae87-c3d5a824f42e None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:48.340882 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 655/1321] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:52:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:52:48.346914 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723d2a1-17d2-4de1-8482-3261d6758642 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:48.347346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723d2a1-17d2-4de1-8482-3261d6758642 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:48.347524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0723d2a1-17d2-4de1-8482-3261d6758642 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:48.424022 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-0723d2a1-17d2-4de1-8482-3261d6758642 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:52:48.425245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-0723d2a1-17d2-4de1-8482-3261d6758642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:48.487505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:52:48.535126 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 667/1322] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:52:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:52:48.566393 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754681d5-b1d8-48fa-ae87-c3d5a824f42e None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:52:48.568494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-754681d5-b1d8-48fa-ae87-c3d5a824f42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:48.588091 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:52:48.673316 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754681d5-b1d8-48fa-ae87-c3d5a824f42e None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:52:48.746123 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d810e9e-e3fa-4442-8c30-e441594b544a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:52:48.746744 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d810e9e-e3fa-4442-8c30-e441594b544a service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:48.747030 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d810e9e-e3fa-4442-8c30-e441594b544a service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:48.747300 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d810e9e-e3fa-4442-8c30-e441594b544a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:48.748928 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-3d810e9e-e3fa-4442-8c30-e441594b544a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:48.820005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:52:48.858363 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d810e9e-e3fa-4442-8c30-e441594b544a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:52:48.859847 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d810e9e-e3fa-4442-8c30-e441594b544a service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:52:48.878145 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 656/1323] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:52:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:52:51.067099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5998e9-9545-43eb-94e8-96efc2960f2f None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:51.067607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5998e9-9545-43eb-94e8-96efc2960f2f None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:51.067607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5998e9-9545-43eb-94e8-96efc2960f2f None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:51.068939 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 668/1324] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:52:51.073990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a7cd7-6e9c-41e4-96bc-e3e51c2cd2f7 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:51.074387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a7cd7-6e9c-41e4-96bc-e3e51c2cd2f7 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:51.074551 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379a7cd7-6e9c-41e4-96bc-e3e51c2cd2f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:51.138620 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-379a7cd7-6e9c-41e4-96bc-e3e51c2cd2f7 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:52:51.139814 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-379a7cd7-6e9c-41e4-96bc-e3e51c2cd2f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:51.237001 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 657/1325] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:52:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 22:52:51.273205 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5998e9-9545-43eb-94e8-96efc2960f2f None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:52:51.274771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-cc5998e9-9545-43eb-94e8-96efc2960f2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:51.381500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5998e9-9545-43eb-94e8-96efc2960f2f None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:52:51.453366 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f900a85b-a6e9-4571-a258-85b1f4ab0c87 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:52:51.454015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900a85b-a6e9-4571-a258-85b1f4ab0c87 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:51.454300 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900a85b-a6e9-4571-a258-85b1f4ab0c87 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:51.454573 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f900a85b-a6e9-4571-a258-85b1f4ab0c87 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:51.456215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-f900a85b-a6e9-4571-a258-85b1f4ab0c87 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:51.474500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:52:51.577775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900a85b-a6e9-4571-a258-85b1f4ab0c87 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:52:51.579238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f900a85b-a6e9-4571-a258-85b1f4ab0c87 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:52:51.597393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 669/1326] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:52:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:52:53.788101 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdd4b2-d92b-469a-9e99-113665193894 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:53.788626 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdd4b2-d92b-469a-9e99-113665193894 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:53.788718 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfdd4b2-d92b-469a-9e99-113665193894 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:53.790027 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 658/1327] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:52:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:52:53.795075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d41752-f9d7-4588-b6f8-41039d8ab27a None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:53.795455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d41752-f9d7-4588-b6f8-41039d8ab27a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:53.795640 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d41752-f9d7-4588-b6f8-41039d8ab27a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:53.861284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-47d41752-f9d7-4588-b6f8-41039d8ab27a None None] MFA Rules not processed for user `506c986a39184b05b22b043ea6c2f174`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:52:53.862501 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-47d41752-f9d7-4588-b6f8-41039d8ab27a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:53.935288 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:52:53.960206 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 670/1328] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:52:53 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:52:54.011705 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1184eb-a86b-4070-a209-7bd8ad7b87fd None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:54.012091 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1184eb-a86b-4070-a209-7bd8ad7b87fd None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:54.012476 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1184eb-a86b-4070-a209-7bd8ad7b87fd None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:54.013934 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 659/1329] 10.222.0.17 () {54 vars in 964 bytes} [Thu Jun 16 22:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:52:54.019836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fed5c2-3c72-4558-8776-ed75f324e4f0 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:54.020127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fed5c2-3c72-4558-8776-ed75f324e4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:54.020951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fed5c2-3c72-4558-8776-ed75f324e4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:54.104701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-41fed5c2-3c72-4558-8776-ed75f324e4f0 None None] MFA Rules not processed for user `6b794f7aa2a7488e91f7d9ea35a45d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:52:54.105901 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-41fed5c2-3c72-4558-8776-ed75f324e4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:54.186478 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 671/1330] 10.222.0.17 () {58 vars in 1061 bytes} [Thu Jun 16 22:52:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:52:54.195637 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1184eb-a86b-4070-a209-7bd8ad7b87fd None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:52:54.197287 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-8d1184eb-a86b-4070-a209-7bd8ad7b87fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:54.229137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:52:54.300789 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1184eb-a86b-4070-a209-7bd8ad7b87fd None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:52:54.373715 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf86345-1cd5-476c-915a-553463bc85c7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:52:54.374372 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf86345-1cd5-476c-915a-553463bc85c7 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:54.374755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf86345-1cd5-476c-915a-553463bc85c7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:54.375036 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf86345-1cd5-476c-915a-553463bc85c7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:54.376664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6cf86345-1cd5-476c-915a-553463bc85c7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:54.455213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:52:54.486794 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf86345-1cd5-476c-915a-553463bc85c7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:52:54.488225 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf86345-1cd5-476c-915a-553463bc85c7 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:52:54.513640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 660/1331] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:52:54 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 22:52:57.271806 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d2cdd-4bb7-4417-9cd5-19573d4187fc None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:57.272354 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d2cdd-4bb7-4417-9cd5-19573d4187fc None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:57.272354 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d2cdd-4bb7-4417-9cd5-19573d4187fc None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:57.273743 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 672/1332] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:52:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:52:57.278849 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4342601-d816-4754-b537-79d230755624 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:57.279226 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4342601-d816-4754-b537-79d230755624 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:57.279417 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4342601-d816-4754-b537-79d230755624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:57.342673 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-d4342601-d816-4754-b537-79d230755624 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:52:57.343895 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d4342601-d816-4754-b537-79d230755624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:57.438189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 661/1333] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:52:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:52:57.473639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d2cdd-4bb7-4417-9cd5-19573d4187fc None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:52:57.475291 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-db3d2cdd-4bb7-4417-9cd5-19573d4187fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:57.546661 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:52:57.580233 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d2cdd-4bb7-4417-9cd5-19573d4187fc None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:52:57.651820 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb81d0f-d9e8-42d3-a43e-928ffbf2e6a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:52:57.652468 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb81d0f-d9e8-42d3-a43e-928ffbf2e6a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:57.652757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb81d0f-d9e8-42d3-a43e-928ffbf2e6a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:57.653038 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb81d0f-d9e8-42d3-a43e-928ffbf2e6a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:57.654694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-9bb81d0f-d9e8-42d3-a43e-928ffbf2e6a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:52:57.765343 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb81d0f-d9e8-42d3-a43e-928ffbf2e6a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:52:57.766797 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb81d0f-d9e8-42d3-a43e-928ffbf2e6a7 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:52:57.777286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:52:57.786183 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 673/1334] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:52:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:52:59.931102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec343a11-b44a-443d-b503-05e7f0d58a30 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:59.931650 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec343a11-b44a-443d-b503-05e7f0d58a30 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:59.931650 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec343a11-b44a-443d-b503-05e7f0d58a30 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:52:59.933025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 662/1335] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:52:59.938177 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27deafb3-606a-441c-bbf5-31d260adb4fb None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:52:59.938561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27deafb3-606a-441c-bbf5-31d260adb4fb None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:52:59.938791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27deafb3-606a-441c-bbf5-31d260adb4fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:00.007164 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-27deafb3-606a-441c-bbf5-31d260adb4fb None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:00.008372 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-27deafb3-606a-441c-bbf5-31d260adb4fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:00.102517 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 674/1336] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:52:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:00.142680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec343a11-b44a-443d-b503-05e7f0d58a30 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:00.144189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-ec343a11-b44a-443d-b503-05e7f0d58a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:00.183527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:53:00.246452 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec343a11-b44a-443d-b503-05e7f0d58a30 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:00.319698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d934b197-1f34-4b77-8c03-2a59c8e344fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:00.320354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934b197-1f34-4b77-8c03-2a59c8e344fa service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:00.320646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934b197-1f34-4b77-8c03-2a59c8e344fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:00.320923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d934b197-1f34-4b77-8c03-2a59c8e344fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:00.322562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d934b197-1f34-4b77-8c03-2a59c8e344fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:00.404302 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:53:00.430924 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934b197-1f34-4b77-8c03-2a59c8e344fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:00.433175 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d934b197-1f34-4b77-8c03-2a59c8e344fa service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:00.452089 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 663/1337] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 22:53:03.040901 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8fb8c-68eb-4716-abc6-56b81eeb065a None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:03.041425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8fb8c-68eb-4716-abc6-56b81eeb065a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:03.041425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e8fb8c-68eb-4716-abc6-56b81eeb065a None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:03.042739 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 675/1338] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:03.047823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5cdfc-155f-40b3-af38-73309ebdd8a2 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:03.048227 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5cdfc-155f-40b3-af38-73309ebdd8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:03.048381 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef5cdfc-155f-40b3-af38-73309ebdd8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:03.111261 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-3ef5cdfc-155f-40b3-af38-73309ebdd8a2 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:03.112467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-3ef5cdfc-155f-40b3-af38-73309ebdd8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:03.206422 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 664/1339] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:53:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:03.241571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8fb8c-68eb-4716-abc6-56b81eeb065a None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:03.243138 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b7e8fb8c-68eb-4716-abc6-56b81eeb065a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:03.321713 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:53:03.352912 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e8fb8c-68eb-4716-abc6-56b81eeb065a None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:03.424009 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a6df62-1545-4194-a09c-1cb7459bb4d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:03.424652 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6df62-1545-4194-a09c-1cb7459bb4d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:03.424938 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6df62-1545-4194-a09c-1cb7459bb4d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:03.425215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a6df62-1545-4194-a09c-1cb7459bb4d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:03.427411 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b6a6df62-1545-4194-a09c-1cb7459bb4d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:03.532480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6df62-1545-4194-a09c-1cb7459bb4d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:03.533905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a6df62-1545-4194-a09c-1cb7459bb4d4 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:03.552494 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 676/1340] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:53:07.633997 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb479761-8527-47a5-9b08-eba797a40079 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:07.634570 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb479761-8527-47a5-9b08-eba797a40079 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:07.634570 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb479761-8527-47a5-9b08-eba797a40079 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:07.635962 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 665/1341] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:07.641128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd629111-5d30-4493-a24f-c830db122970 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:07.641546 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd629111-5d30-4493-a24f-c830db122970 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:07.641699 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd629111-5d30-4493-a24f-c830db122970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:07.648721 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:53:07.713969 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-fd629111-5d30-4493-a24f-c830db122970 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:07.715157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-fd629111-5d30-4493-a24f-c830db122970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:07.819442 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 677/1342] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:53:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:07.857219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb479761-8527-47a5-9b08-eba797a40079 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:07.859550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-eb479761-8527-47a5-9b08-eba797a40079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:07.910257 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:53:07.954917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb479761-8527-47a5-9b08-eba797a40079 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:07.958450 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3963817b-2f4d-4905-be69-3d19d902deba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:07.959130 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963817b-2f4d-4905-be69-3d19d902deba service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:07.959412 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963817b-2f4d-4905-be69-3d19d902deba service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:07.959689 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3963817b-2f4d-4905-be69-3d19d902deba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:07.968580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3963817b-2f4d-4905-be69-3d19d902deba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:07.969996 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3963817b-2f4d-4905-be69-3d19d902deba service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:07.979822 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 666/1343] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:53:10.693284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d538ba-d4ed-40d9-98d5-73c2e62ca622 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:10.693836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d538ba-d4ed-40d9-98d5-73c2e62ca622 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:10.693836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d538ba-d4ed-40d9-98d5-73c2e62ca622 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:10.695230 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 678/1344] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:10.700492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f350e68-71d4-4ffa-a505-841eeb44860d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:10.700950 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f350e68-71d4-4ffa-a505-841eeb44860d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:10.701108 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f350e68-71d4-4ffa-a505-841eeb44860d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:10.731034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-4f350e68-71d4-4ffa-a505-841eeb44860d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:10.732273 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-4f350e68-71d4-4ffa-a505-841eeb44860d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:10.814050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 667/1345] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:53:10 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:10.851842 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d538ba-d4ed-40d9-98d5-73c2e62ca622 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:10.863416 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d538ba-d4ed-40d9-98d5-73c2e62ca622 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:10.866988 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9b415d-ffac-465e-832c-8618c16256fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:10.867647 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b415d-ffac-465e-832c-8618c16256fc service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:10.867932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b415d-ffac-465e-832c-8618c16256fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:10.868210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9b415d-ffac-465e-832c-8618c16256fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:10.877822 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b415d-ffac-465e-832c-8618c16256fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:10.879949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9b415d-ffac-465e-832c-8618c16256fc service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:10.889251 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 679/1346] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:10 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:53:14.897995 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836b011-ceca-426c-97ca-f4178105f532 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:14.898482 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836b011-ceca-426c-97ca-f4178105f532 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:14.898619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0836b011-ceca-426c-97ca-f4178105f532 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:14.899953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 668/1347] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:14.905144 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be62d6-85e7-45ce-9b4a-743bec43c7c1 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:14.905529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be62d6-85e7-45ce-9b4a-743bec43c7c1 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:14.905731 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be62d6-85e7-45ce-9b4a-743bec43c7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:14.924379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-22be62d6-85e7-45ce-9b4a-743bec43c7c1 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:14.926216 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-22be62d6-85e7-45ce-9b4a-743bec43c7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:14.934705 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 680/1348] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:53:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:14.972302 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836b011-ceca-426c-97ca-f4178105f532 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:14.983050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0836b011-ceca-426c-97ca-f4178105f532 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:14.986674 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2543b-3913-485e-8737-1fbd186ff311 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:14.987359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2543b-3913-485e-8737-1fbd186ff311 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:14.987644 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2543b-3913-485e-8737-1fbd186ff311 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:14.987921 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2543b-3913-485e-8737-1fbd186ff311 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:14.998449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2543b-3913-485e-8737-1fbd186ff311 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:15.000026 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2543b-3913-485e-8737-1fbd186ff311 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:15.011032 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 669/1349] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:53:20.129767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f4316-a23e-4e2c-a984-05d5162547f0 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:20.130925 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f4316-a23e-4e2c-a984-05d5162547f0 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:20.131392 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4f4316-a23e-4e2c-a984-05d5162547f0 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:20.133394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 681/1350] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 22:53:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:20.138661 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff22f4b-43ec-40e1-9d8d-4b50605743af None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:20.139078 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff22f4b-43ec-40e1-9d8d-4b50605743af None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:20.139303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff22f4b-43ec-40e1-9d8d-4b50605743af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:20.218721 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-0ff22f4b-43ec-40e1-9d8d-4b50605743af None None] MFA Rules not processed for user `f9593939641c4c7797c1981206b40e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:20.220469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-0ff22f4b-43ec-40e1-9d8d-4b50605743af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:20.258994 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:53:20.325147 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 670/1351] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 22:53:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:24.481057 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd55c46-0f33-4e2a-aae2-cf3d2d98fc1f None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:24.481706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd55c46-0f33-4e2a-aae2-cf3d2d98fc1f None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:24.481706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd55c46-0f33-4e2a-aae2-cf3d2d98fc1f None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:24.483203 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 682/1352] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 22:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:24.488243 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0691bda-65a2-47e3-864d-551ea9279b87 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:24.488672 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0691bda-65a2-47e3-864d-551ea9279b87 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:24.488769 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0691bda-65a2-47e3-864d-551ea9279b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:24.508877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-f0691bda-65a2-47e3-864d-551ea9279b87 None None] MFA Rules not processed for user `f9593939641c4c7797c1981206b40e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:24.510145 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f0691bda-65a2-47e3-864d-551ea9279b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:24.519838 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 671/1353] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 22:53:24 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:24.918353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=72 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:53:25.913614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4f917-5c69-41a7-81e6-74b19c6fc503 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:25.914272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4f917-5c69-41a7-81e6-74b19c6fc503 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:25.914272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f4f917-5c69-41a7-81e6-74b19c6fc503 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:25.915607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 683/1354] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 22:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:25.920696 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0d112-051c-45f9-be82-ea5429a901a5 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:25.921723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0d112-051c-45f9-be82-ea5429a901a5 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:25.922031 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0d112-051c-45f9-be82-ea5429a901a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:25.940668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-c9a0d112-051c-45f9-be82-ea5429a901a5 None None] MFA Rules not processed for user `f9593939641c4c7797c1981206b40e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:25.941865 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c9a0d112-051c-45f9-be82-ea5429a901a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:25.950666 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 672/1355] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 22:53:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:29.787242 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86fe07-2986-4c11-9fd9-91386dfcfb23 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:29.787761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86fe07-2986-4c11-9fd9-91386dfcfb23 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:29.787761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e86fe07-2986-4c11-9fd9-91386dfcfb23 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:29.789032 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 684/1356] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:29.794153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540473e-948d-4f58-975e-5740ee66afe6 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:29.794567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540473e-948d-4f58-975e-5740ee66afe6 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:29.794747 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5540473e-948d-4f58-975e-5740ee66afe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:29.812232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-5540473e-948d-4f58-975e-5740ee66afe6 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:29.813399 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-5540473e-948d-4f58-975e-5740ee66afe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:29.821827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 673/1357] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:53:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:29.873888 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6ec5d-a29b-43b8-a7b6-d041645e4f4c None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:29.874393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6ec5d-a29b-43b8-a7b6-d041645e4f4c None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:29.875120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6ec5d-a29b-43b8-a7b6-d041645e4f4c None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:29.877675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 685/1358] 10.222.0.17 () {54 vars in 961 bytes} [Thu Jun 16 22:53:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:29.885245 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39560a2f-7495-4261-bbe5-18440ddf7e11 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:29.885740 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39560a2f-7495-4261-bbe5-18440ddf7e11 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:29.886168 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39560a2f-7495-4261-bbe5-18440ddf7e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:29.940721 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-39560a2f-7495-4261-bbe5-18440ddf7e11 None None] MFA Rules not processed for user `4180f2593149437fa27effd8d1338d10`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:29.941897 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-39560a2f-7495-4261-bbe5-18440ddf7e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:30.025364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 674/1359] 10.222.0.17 () {58 vars in 1058 bytes} [Thu Jun 16 22:53:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 22:53:30.033234 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6ec5d-a29b-43b8-a7b6-d041645e4f4c None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:30.047384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d6ec5d-a29b-43b8-a7b6-d041645e4f4c None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:30.053975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f3e51a-7337-4767-a216-58c6fa782eb0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:30.054630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3e51a-7337-4767-a216-58c6fa782eb0 service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:30.054923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3e51a-7337-4767-a216-58c6fa782eb0 service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:30.055199 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f3e51a-7337-4767-a216-58c6fa782eb0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:30.063570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f3e51a-7337-4767-a216-58c6fa782eb0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:30.065703 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f3e51a-7337-4767-a216-58c6fa782eb0 service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:30.075352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 686/1360] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:53:32.428131 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4566f6-4b79-4804-8a08-b97a3f72c037 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:32.428721 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4566f6-4b79-4804-8a08-b97a3f72c037 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:32.428721 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4566f6-4b79-4804-8a08-b97a3f72c037 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:32.430321 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 675/1361] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:32.436135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c3c3c-1e07-4ace-b873-3a6a53b83b2a None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:32.436685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c3c3c-1e07-4ace-b873-3a6a53b83b2a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:32.436685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c3c3c-1e07-4ace-b873-3a6a53b83b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:32.456910 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-323c3c3c-1e07-4ace-b873-3a6a53b83b2a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:32.458528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-323c3c3c-1e07-4ace-b873-3a6a53b83b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:32.472606 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 687/1362] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:53:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:32.516628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4566f6-4b79-4804-8a08-b97a3f72c037 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:32.536298 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4566f6-4b79-4804-8a08-b97a3f72c037 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:32.543144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b534d8-10df-4e07-bc4d-b868b9652281 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:32.544350 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b534d8-10df-4e07-bc4d-b868b9652281 service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:32.544739 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b534d8-10df-4e07-bc4d-b868b9652281 service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:32.545119 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b534d8-10df-4e07-bc4d-b868b9652281 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:32.554972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b534d8-10df-4e07-bc4d-b868b9652281 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:32.556715 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b534d8-10df-4e07-bc4d-b868b9652281 service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:32.568439 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 676/1363] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:53:35.040227 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7abb62-ab63-4cca-bc40-37feccdc0a64 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:35.040715 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7abb62-ab63-4cca-bc40-37feccdc0a64 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:35.040715 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7abb62-ab63-4cca-bc40-37feccdc0a64 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:35.042027 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 688/1364] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:53:35.047279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099ade8-ccc6-447f-9f1b-ed0c2704525d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:35.047719 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099ade8-ccc6-447f-9f1b-ed0c2704525d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:35.047846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4099ade8-ccc6-447f-9f1b-ed0c2704525d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:35.066141 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-4099ade8-ccc6-447f-9f1b-ed0c2704525d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:35.067390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-4099ade8-ccc6-447f-9f1b-ed0c2704525d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:35.075628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 677/1365] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:53:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:35.123529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594964b-7972-4602-8e8c-edc093e84e3a None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:35.123835 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594964b-7972-4602-8e8c-edc093e84e3a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:35.124122 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594964b-7972-4602-8e8c-edc093e84e3a None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:35.125492 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 689/1366] 10.222.0.17 () {54 vars in 961 bytes} [Thu Jun 16 22:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:35.131060 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981a091-19fa-4e78-bddb-1fc92cc1e2a8 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:35.131356 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981a091-19fa-4e78-bddb-1fc92cc1e2a8 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:35.131642 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981a091-19fa-4e78-bddb-1fc92cc1e2a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:35.150568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-2981a091-19fa-4e78-bddb-1fc92cc1e2a8 None None] MFA Rules not processed for user `4180f2593149437fa27effd8d1338d10`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:35.151824 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-2981a091-19fa-4e78-bddb-1fc92cc1e2a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:35.161797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 678/1367] 10.222.0.17 () {58 vars in 1058 bytes} [Thu Jun 16 22:53:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 22:53:35.169880 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594964b-7972-4602-8e8c-edc093e84e3a None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:35.186063 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f594964b-7972-4602-8e8c-edc093e84e3a None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:35.191597 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc8e638-37f5-46c5-b4c5-8f318abd1367 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:35.192186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8e638-37f5-46c5-b4c5-8f318abd1367 service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:35.192473 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8e638-37f5-46c5-b4c5-8f318abd1367 service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:35.192751 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc8e638-37f5-46c5-b4c5-8f318abd1367 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:35.201002 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8e638-37f5-46c5-b4c5-8f318abd1367 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:35.202569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc8e638-37f5-46c5-b4c5-8f318abd1367 service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:35.213240 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 690/1368] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:53:37.403690 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2cfba-a8b7-4359-ab2f-e7d0ec48784e None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:37.404668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2cfba-a8b7-4359-ab2f-e7d0ec48784e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:37.405175 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b2cfba-a8b7-4359-ab2f-e7d0ec48784e None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:37.408298 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 679/1369] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:53:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:37.413135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba0e50-3afc-4433-aac7-142b7a9c2f02 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:37.414056 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba0e50-3afc-4433-aac7-142b7a9c2f02 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:37.414531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ba0e50-3afc-4433-aac7-142b7a9c2f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:37.434919 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-33ba0e50-3afc-4433-aac7-142b7a9c2f02 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:37.437108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-33ba0e50-3afc-4433-aac7-142b7a9c2f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:37.447842 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 691/1370] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:53:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:37.492885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2cfba-a8b7-4359-ab2f-e7d0ec48784e None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:37.509992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b2cfba-a8b7-4359-ab2f-e7d0ec48784e None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:37.515958 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2db304-7404-46ef-8e51-0f4d031bfabd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:37.516625 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2db304-7404-46ef-8e51-0f4d031bfabd service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:37.516910 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2db304-7404-46ef-8e51-0f4d031bfabd service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:37.517187 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2db304-7404-46ef-8e51-0f4d031bfabd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:37.527253 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2db304-7404-46ef-8e51-0f4d031bfabd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:37.529398 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2db304-7404-46ef-8e51-0f4d031bfabd service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:37.541336 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 680/1371] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:53:37.888825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a2704-a9f7-44a4-abac-5ba0e1b6978d None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:37.889123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a2704-a9f7-44a4-abac-5ba0e1b6978d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:37.889405 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168a2704-a9f7-44a4-abac-5ba0e1b6978d None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:37.890832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 692/1372] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:37.896679 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946db2f-0ba8-42ae-9de0-286d9c943624 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:37.896998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946db2f-0ba8-42ae-9de0-286d9c943624 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:37.897279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946db2f-0ba8-42ae-9de0-286d9c943624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:37.916511 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-a946db2f-0ba8-42ae-9de0-286d9c943624 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:37.917699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-a946db2f-0ba8-42ae-9de0-286d9c943624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:37.926641 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 681/1373] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:53:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 22:53:38.251815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07758f40-7692-4ab5-8f7d-eb96607fda7f None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:38.252397 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07758f40-7692-4ab5-8f7d-eb96607fda7f None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:38.252907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07758f40-7692-4ab5-8f7d-eb96607fda7f None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:38.255286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 693/1374] 10.222.0.17 () {54 vars in 963 bytes} [Thu Jun 16 22:53:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:38.260513 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b8c9a-10bf-482a-b3a6-050aba77d9f1 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:38.260813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b8c9a-10bf-482a-b3a6-050aba77d9f1 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:38.261096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5b8c9a-10bf-482a-b3a6-050aba77d9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:38.269522 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:53:38.314709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-2d5b8c9a-10bf-482a-b3a6-050aba77d9f1 None None] MFA Rules not processed for user `24ca7f221334442781f10574277bf422`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:38.315904 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-2d5b8c9a-10bf-482a-b3a6-050aba77d9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:38.380852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 682/1375] 10.222.0.17 () {58 vars in 1060 bytes} [Thu Jun 16 22:53:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:38.388903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07758f40-7692-4ab5-8f7d-eb96607fda7f None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:38.401817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07758f40-7692-4ab5-8f7d-eb96607fda7f None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:38.405872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bdf402-adc5-4454-99b0-43196925a7c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:38.406818 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdf402-adc5-4454-99b0-43196925a7c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:38.407624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdf402-adc5-4454-99b0-43196925a7c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:38.408591 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bdf402-adc5-4454-99b0-43196925a7c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:38.419049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bdf402-adc5-4454-99b0-43196925a7c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:38.421289 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bdf402-adc5-4454-99b0-43196925a7c0 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:38.432415 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 694/1376] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:53:39.741999 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef68bbe-d802-4e4b-8e74-7fb0fda10c92 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:39.742639 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef68bbe-d802-4e4b-8e74-7fb0fda10c92 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:39.742639 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef68bbe-d802-4e4b-8e74-7fb0fda10c92 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:39.744037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 683/1377] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:39.749330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755967b2-b6f2-45a9-9661-0e3a6354edee None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:39.749753 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755967b2-b6f2-45a9-9661-0e3a6354edee None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:39.749894 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755967b2-b6f2-45a9-9661-0e3a6354edee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:39.769065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-755967b2-b6f2-45a9-9661-0e3a6354edee None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:39.770520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-755967b2-b6f2-45a9-9661-0e3a6354edee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:39.778384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 695/1378] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:53:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:39.823876 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef68bbe-d802-4e4b-8e74-7fb0fda10c92 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:39.838658 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ef68bbe-d802-4e4b-8e74-7fb0fda10c92 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:39.843942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d78ba27-8748-4894-b8de-a7c54d4692fd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:39.844653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78ba27-8748-4894-b8de-a7c54d4692fd service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:39.845441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78ba27-8748-4894-b8de-a7c54d4692fd service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:39.845717 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d78ba27-8748-4894-b8de-a7c54d4692fd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:39.854016 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78ba27-8748-4894-b8de-a7c54d4692fd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:39.855622 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d78ba27-8748-4894-b8de-a7c54d4692fd service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:39.865473 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 684/1379] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:53:42.098459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e3601-d210-4049-9b80-40a5176881b9 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:42.099009 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e3601-d210-4049-9b80-40a5176881b9 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:42.099009 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e3601-d210-4049-9b80-40a5176881b9 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:42.100300 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 696/1380] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:42.105484 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46267c-77b8-4965-9f04-e8a2361a2473 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:42.105913 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46267c-77b8-4965-9f04-e8a2361a2473 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:42.106047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc46267c-77b8-4965-9f04-e8a2361a2473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:42.125183 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-cc46267c-77b8-4965-9f04-e8a2361a2473 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:42.127034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-cc46267c-77b8-4965-9f04-e8a2361a2473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:42.137898 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 685/1381] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:53:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:42.179951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e3601-d210-4049-9b80-40a5176881b9 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:42.196298 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e3601-d210-4049-9b80-40a5176881b9 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:42.201977 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff14b20-9ea8-4cd8-b1aa-488755d1c708 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:42.202669 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff14b20-9ea8-4cd8-b1aa-488755d1c708 service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:42.202965 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff14b20-9ea8-4cd8-b1aa-488755d1c708 service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:42.203248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff14b20-9ea8-4cd8-b1aa-488755d1c708 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:42.212695 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff14b20-9ea8-4cd8-b1aa-488755d1c708 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:42.214287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff14b20-9ea8-4cd8-b1aa-488755d1c708 service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:42.224531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 697/1382] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:53:44.433811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd658793-b2c3-40b3-8552-0eb8d4ea372d None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:44.434356 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd658793-b2c3-40b3-8552-0eb8d4ea372d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:44.434356 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd658793-b2c3-40b3-8552-0eb8d4ea372d None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:44.435953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 686/1383] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:44.441154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad406b-ad56-4161-8202-3212cc7b9871 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:44.441599 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad406b-ad56-4161-8202-3212cc7b9871 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:44.441724 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad406b-ad56-4161-8202-3212cc7b9871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:44.460774 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-f9ad406b-ad56-4161-8202-3212cc7b9871 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:44.461964 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-f9ad406b-ad56-4161-8202-3212cc7b9871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:44.470722 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 698/1384] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:53:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:44.512279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd658793-b2c3-40b3-8552-0eb8d4ea372d None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:44.527967 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd658793-b2c3-40b3-8552-0eb8d4ea372d None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:44.533951 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc5bfc7-332a-4f5a-918c-acb0935a4492 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:44.534616 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5bfc7-332a-4f5a-918c-acb0935a4492 service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:44.534931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5bfc7-332a-4f5a-918c-acb0935a4492 service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:44.535203 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc5bfc7-332a-4f5a-918c-acb0935a4492 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:44.544182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5bfc7-332a-4f5a-918c-acb0935a4492 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:44.545778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc5bfc7-332a-4f5a-918c-acb0935a4492 service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:44.555782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 687/1385] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:53:46.722090 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a9433-02b6-4a0b-9c24-4f6d5aaf2f36 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:46.722725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a9433-02b6-4a0b-9c24-4f6d5aaf2f36 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:46.722864 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13a9433-02b6-4a0b-9c24-4f6d5aaf2f36 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:46.724642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 699/1386] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:53:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:46.729828 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcdee9-7d17-4415-acad-4e4ef2f7df1d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:46.730238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcdee9-7d17-4415-acad-4e4ef2f7df1d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:46.730389 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabcdee9-7d17-4415-acad-4e4ef2f7df1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:46.749190 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-dabcdee9-7d17-4415-acad-4e4ef2f7df1d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:46.751072 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-dabcdee9-7d17-4415-acad-4e4ef2f7df1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:46.760795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 688/1387] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:53:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:46.805714 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a9433-02b6-4a0b-9c24-4f6d5aaf2f36 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:46.820903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13a9433-02b6-4a0b-9c24-4f6d5aaf2f36 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:46.826543 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3be59a5-8c52-4237-8f5d-a66a0e220497 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:46.827261 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be59a5-8c52-4237-8f5d-a66a0e220497 service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:46.827553 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be59a5-8c52-4237-8f5d-a66a0e220497 service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:46.827833 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3be59a5-8c52-4237-8f5d-a66a0e220497 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:46.836547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be59a5-8c52-4237-8f5d-a66a0e220497 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:46.838124 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3be59a5-8c52-4237-8f5d-a66a0e220497 service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:46.848940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 700/1388] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:53:49.043115 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7e11f-0070-4b1d-b9ac-aac59d35879c None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:49.043676 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7e11f-0070-4b1d-b9ac-aac59d35879c None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:49.043676 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51c7e11f-0070-4b1d-b9ac-aac59d35879c None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:49.045052 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 689/1389] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:49.051905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2791e3c-4fd2-4194-92eb-19ae855ec30a None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:49.052476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2791e3c-4fd2-4194-92eb-19ae855ec30a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:49.053013 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2791e3c-4fd2-4194-92eb-19ae855ec30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:49.085408 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-c2791e3c-4fd2-4194-92eb-19ae855ec30a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:49.087706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c2791e3c-4fd2-4194-92eb-19ae855ec30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:49.106627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 701/1390] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:53:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:49.152204 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7e11f-0070-4b1d-b9ac-aac59d35879c None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:49.167353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51c7e11f-0070-4b1d-b9ac-aac59d35879c None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:49.173885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7ddad-9384-4b64-8ae8-bbc9825ff358 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:49.174574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7ddad-9384-4b64-8ae8-bbc9825ff358 service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:49.174901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7ddad-9384-4b64-8ae8-bbc9825ff358 service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:49.175169 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7ddad-9384-4b64-8ae8-bbc9825ff358 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:49.184953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7ddad-9384-4b64-8ae8-bbc9825ff358 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:49.186451 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7ddad-9384-4b64-8ae8-bbc9825ff358 service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:49.197349 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 690/1391] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:53:51.362463 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c75943-fd98-4a1f-86dc-f271de506903 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:51.363057 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c75943-fd98-4a1f-86dc-f271de506903 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:51.363178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c75943-fd98-4a1f-86dc-f271de506903 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:51.364468 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 702/1392] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:51.370401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70a227-d502-4ec6-8f44-b0206d730bb2 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:51.370859 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70a227-d502-4ec6-8f44-b0206d730bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:51.371180 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b70a227-d502-4ec6-8f44-b0206d730bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:51.393838 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-3b70a227-d502-4ec6-8f44-b0206d730bb2 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:51.395419 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-3b70a227-d502-4ec6-8f44-b0206d730bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:51.406721 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 691/1393] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:53:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:51.459875 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c75943-fd98-4a1f-86dc-f271de506903 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:51.474558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c75943-fd98-4a1f-86dc-f271de506903 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:51.480043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17c376-d4e5-4322-8e9f-542f8029dd7d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:51.480684 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17c376-d4e5-4322-8e9f-542f8029dd7d service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:51.480964 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17c376-d4e5-4322-8e9f-542f8029dd7d service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:51.481234 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17c376-d4e5-4322-8e9f-542f8029dd7d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:51.490440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17c376-d4e5-4322-8e9f-542f8029dd7d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:51.491988 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17c376-d4e5-4322-8e9f-542f8029dd7d service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:51.501527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 703/1394] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 22:53:53.687374 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c13bc5-ad2c-4e89-be82-e38f87b9b8f4 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:53.687995 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c13bc5-ad2c-4e89-be82-e38f87b9b8f4 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:53.687995 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c13bc5-ad2c-4e89-be82-e38f87b9b8f4 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:53.689306 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 692/1395] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:53.694417 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13064c45-fec9-40b4-ba69-e1d5e2ede214 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:53.694860 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13064c45-fec9-40b4-ba69-e1d5e2ede214 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:53.695027 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13064c45-fec9-40b4-ba69-e1d5e2ede214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:53.713000 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-13064c45-fec9-40b4-ba69-e1d5e2ede214 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:53.714226 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-13064c45-fec9-40b4-ba69-e1d5e2ede214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:53.722833 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 704/1396] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:53.769029 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c13bc5-ad2c-4e89-be82-e38f87b9b8f4 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:53.787073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c13bc5-ad2c-4e89-be82-e38f87b9b8f4 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:53.792928 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97a372-a547-4f74-a522-507c362296a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:53.793594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97a372-a547-4f74-a522-507c362296a2 service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:53.793889 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97a372-a547-4f74-a522-507c362296a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:53.794178 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97a372-a547-4f74-a522-507c362296a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:53.803759 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97a372-a547-4f74-a522-507c362296a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:53.805350 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97a372-a547-4f74-a522-507c362296a2 service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:53.817382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 693/1397] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:53:53.828293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af2ffc-7beb-4ad3-bf54-433b353f8880 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:53.828594 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af2ffc-7beb-4ad3-bf54-433b353f8880 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:53.828860 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29af2ffc-7beb-4ad3-bf54-433b353f8880 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:53.830239 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 705/1398] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:53.836651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28574eae-9446-4ab9-89d9-47d25bfd6527 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:53.836943 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28574eae-9446-4ab9-89d9-47d25bfd6527 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:53.837219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28574eae-9446-4ab9-89d9-47d25bfd6527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:53.857750 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-28574eae-9446-4ab9-89d9-47d25bfd6527 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:53.859766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-28574eae-9446-4ab9-89d9-47d25bfd6527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:53.867722 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 694/1399] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:53:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:53.931311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd3368-ca77-47e4-999e-c8ecab9130f9 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:53.931607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd3368-ca77-47e4-999e-c8ecab9130f9 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:53.931892 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dd3368-ca77-47e4-999e-c8ecab9130f9 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:53.933268 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 706/1400] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jun 16 22:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:53.938757 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe4e0e-3e9e-44cb-bce5-67285915c7f8 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:53.938958 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe4e0e-3e9e-44cb-bce5-67285915c7f8 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:53.939234 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbe4e0e-3e9e-44cb-bce5-67285915c7f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:53.994524 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-1dbe4e0e-3e9e-44cb-bce5-67285915c7f8 None None] MFA Rules not processed for user `915f4296504b47d796e3a5b99550eb75`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:53.996401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-1dbe4e0e-3e9e-44cb-bce5-67285915c7f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:54.038787 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:53:54.068459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 695/1401] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jun 16 22:53:53 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:54.076123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd3368-ca77-47e4-999e-c8ecab9130f9 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:54.087387 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dd3368-ca77-47e4-999e-c8ecab9130f9 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:54.091175 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496158f1-f36b-492a-8ae6-357d51390305 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915f4296504b47d796e3a5b99550eb75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:54.091855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496158f1-f36b-492a-8ae6-357d51390305 service glance] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:54.092142 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496158f1-f36b-492a-8ae6-357d51390305 service glance] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:54.092543 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496158f1-f36b-492a-8ae6-357d51390305 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:54.102286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496158f1-f36b-492a-8ae6-357d51390305 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:54.103862 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496158f1-f36b-492a-8ae6-357d51390305 service glance] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:54.114063 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 707/1402] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:53:54.828462 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932513d-e722-4404-b786-e5e7bf33cda4 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:54.828930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932513d-e722-4404-b786-e5e7bf33cda4 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:54.829028 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3932513d-e722-4404-b786-e5e7bf33cda4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:54.876731 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-3932513d-e722-4404-b786-e5e7bf33cda4 None None] MFA Rules not processed for user `faf778e713b54a638c1db4a5db6180b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:54.877940 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-3932513d-e722-4404-b786-e5e7bf33cda4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:54.940520 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 696/1403] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 22:53:54 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:54.986508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6c079-7786-40ff-b90a-a7ca98500991 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:54.987024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6c079-7786-40ff-b90a-a7ca98500991 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:54.987124 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6c079-7786-40ff-b90a-a7ca98500991 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:54.988528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 708/1404] 10.222.0.17 () {54 vars in 942 bytes} [Thu Jun 16 22:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:54.994718 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86563e-4783-4b94-9232-282014e6b7d7 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:54.995093 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86563e-4783-4b94-9232-282014e6b7d7 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:54.995499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba86563e-4783-4b94-9232-282014e6b7d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:55.045618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-ba86563e-4783-4b94-9232-282014e6b7d7 None None] MFA Rules not processed for user `bef518c9430e4274b1066241f821f4a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:55.046969 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-ba86563e-4783-4b94-9232-282014e6b7d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:55.118972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 697/1405] 10.222.0.17 () {58 vars in 1039 bytes} [Thu Jun 16 22:53:54 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:53:55.128761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6c079-7786-40ff-b90a-a7ca98500991 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:55.136400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=61 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:53:55.143527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6c079-7786-40ff-b90a-a7ca98500991 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:55.148442 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5185a185-df96-40ce-94cf-9b790dcbefcd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef518c9430e4274b1066241f821f4a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:55.149094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185a185-df96-40ce-94cf-9b790dcbefcd service swift] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:55.149381 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185a185-df96-40ce-94cf-9b790dcbefcd service swift] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:55.149671 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5185a185-df96-40ce-94cf-9b790dcbefcd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:55.158317 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185a185-df96-40ce-94cf-9b790dcbefcd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:55.159937 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5185a185-df96-40ce-94cf-9b790dcbefcd service swift] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:55.167830 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 709/1406] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 22:53:55 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 22:53:56.016182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec194755-92c6-4f5e-ab9b-bd80a06cae33 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:56.017303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec194755-92c6-4f5e-ab9b-bd80a06cae33 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:56.017303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec194755-92c6-4f5e-ab9b-bd80a06cae33 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:56.019116 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 698/1407] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:53:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:53:56.024312 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07c581-b808-46bc-821b-2da1123bb98e None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:56.025204 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07c581-b808-46bc-821b-2da1123bb98e None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:56.025723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07c581-b808-46bc-821b-2da1123bb98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:56.044322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-ae07c581-b808-46bc-821b-2da1123bb98e None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:53:56.045898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ae07c581-b808-46bc-821b-2da1123bb98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:53:56.054022 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 710/1408] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:53:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 22:53:56.099425 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec194755-92c6-4f5e-ab9b-bd80a06cae33 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:53:56.114440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec194755-92c6-4f5e-ab9b-bd80a06cae33 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:53:56.120141 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121b2ff1-4811-4d38-ac24-ac5b31dc052a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:53:56.120904 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b2ff1-4811-4d38-ac24-ac5b31dc052a service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:53:56.121194 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b2ff1-4811-4d38-ac24-ac5b31dc052a service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:53:56.121472 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121b2ff1-4811-4d38-ac24-ac5b31dc052a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:53:56.131113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b2ff1-4811-4d38-ac24-ac5b31dc052a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:53:56.132824 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121b2ff1-4811-4d38-ac24-ac5b31dc052a service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:53:56.137868 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:53:56.150795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 699/1409] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:53:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:06.038837 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=8 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:54:17.341207 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242a775-51d3-46b9-bc8f-d69ea718450d None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:17.341917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242a775-51d3-46b9-bc8f-d69ea718450d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:17.341917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f242a775-51d3-46b9-bc8f-d69ea718450d None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:17.343162 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 711/1410] 10.222.0.17 () {54 vars in 918 bytes} [Thu Jun 16 22:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:17.348276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0dad5a-3ed8-4d99-a782-79d6440cb943 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:17.348675 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0dad5a-3ed8-4d99-a782-79d6440cb943 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:17.348903 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0dad5a-3ed8-4d99-a782-79d6440cb943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:17.367361 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-0b0dad5a-3ed8-4d99-a782-79d6440cb943 None None] MFA Rules not processed for user `f9593939641c4c7797c1981206b40e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:17.368559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-0b0dad5a-3ed8-4d99-a782-79d6440cb943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:17.377604 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 700/1411] 10.222.0.17 () {58 vars in 1015 bytes} [Thu Jun 16 22:54:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:17.391624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a207258-a87b-4b59-91f2-5d267e8f0c8d None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:17.391915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a207258-a87b-4b59-91f2-5d267e8f0c8d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:17.392191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a207258-a87b-4b59-91f2-5d267e8f0c8d None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:17.393518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 712/1412] 10.222.0.17 () {54 vars in 918 bytes} [Thu Jun 16 22:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:17.397772 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0c544-3f7d-4a1d-bfd5-7dc3c2864f3f None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:17.398055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0c544-3f7d-4a1d-bfd5-7dc3c2864f3f None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:17.398351 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d0c544-3f7d-4a1d-bfd5-7dc3c2864f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:17.415290 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-80d0c544-3f7d-4a1d-bfd5-7dc3c2864f3f None None] MFA Rules not processed for user `4180f2593149437fa27effd8d1338d10`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:17.416451 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-80d0c544-3f7d-4a1d-bfd5-7dc3c2864f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:17.425907 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 701/1413] 10.222.0.17 () {58 vars in 1015 bytes} [Thu Jun 16 22:54:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:20.022049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d81e7d-31b1-4f87-9d1a-1ce7d82ae9b8 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:20.022565 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d81e7d-31b1-4f87-9d1a-1ce7d82ae9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:20.022565 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d81e7d-31b1-4f87-9d1a-1ce7d82ae9b8 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:20.023980 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 22:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:20.029249 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07494d93-b956-476d-85a8-9d133df2cd8d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:20.029690 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07494d93-b956-476d-85a8-9d133df2cd8d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:20.029805 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07494d93-b956-476d-85a8-9d133df2cd8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:20.047127 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-07494d93-b956-476d-85a8-9d133df2cd8d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:20.048302 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-07494d93-b956-476d-85a8-9d133df2cd8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:20.055856 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 22:54:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:20.098445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d81e7d-31b1-4f87-9d1a-1ce7d82ae9b8 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:20.113754 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d81e7d-31b1-4f87-9d1a-1ce7d82ae9b8 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:20.119495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bd7690-4c07-44c2-bb39-b8810cf76e71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:20.120154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd7690-4c07-44c2-bb39-b8810cf76e71 service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:20.120445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd7690-4c07-44c2-bb39-b8810cf76e71 service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:20.120765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bd7690-4c07-44c2-bb39-b8810cf76e71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:20.130141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd7690-4c07-44c2-bb39-b8810cf76e71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:20.131749 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bd7690-4c07-44c2-bb39-b8810cf76e71 service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:20.141993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 714/1416] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:21.099597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc85da-f6b3-48f4-b3ab-df02f2e8b22b None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:21.100131 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc85da-f6b3-48f4-b3ab-df02f2e8b22b None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:21.100255 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccc85da-f6b3-48f4-b3ab-df02f2e8b22b None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:21.101541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 22:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:21.106553 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e508a-0d0c-4e33-afb5-1991d5fc11dc None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:21.107000 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e508a-0d0c-4e33-afb5-1991d5fc11dc None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:21.107133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570e508a-0d0c-4e33-afb5-1991d5fc11dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:21.124665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-570e508a-0d0c-4e33-afb5-1991d5fc11dc None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:21.125837 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-570e508a-0d0c-4e33-afb5-1991d5fc11dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:21.133392 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 22:54:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:21.286991 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc85da-f6b3-48f4-b3ab-df02f2e8b22b None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:21.300960 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccc85da-f6b3-48f4-b3ab-df02f2e8b22b None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:21.306218 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0c94a0-d76f-464a-99e1-4ca7e480222f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:21.306884 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c94a0-d76f-464a-99e1-4ca7e480222f service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:21.307172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c94a0-d76f-464a-99e1-4ca7e480222f service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:21.307453 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0c94a0-d76f-464a-99e1-4ca7e480222f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:21.316102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c94a0-d76f-464a-99e1-4ca7e480222f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:21.317730 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e0c94a0-d76f-464a-99e1-4ca7e480222f service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:21.328078 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 704/1419] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:22.246077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c1ff4-2939-4619-b621-2e156f4af600 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:22.246529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c1ff4-2939-4619-b621-2e156f4af600 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:22.246671 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213c1ff4-2939-4619-b621-2e156f4af600 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:22.248042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 22:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:22.252941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd6b73-50c1-4ffd-93dd-fafaba46ac7d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:22.253375 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd6b73-50c1-4ffd-93dd-fafaba46ac7d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:22.253502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dd6b73-50c1-4ffd-93dd-fafaba46ac7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:22.279297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-41dd6b73-50c1-4ffd-93dd-fafaba46ac7d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:22.280519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-41dd6b73-50c1-4ffd-93dd-fafaba46ac7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:22.288383 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 22:54:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:22.442521 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c1ff4-2939-4619-b621-2e156f4af600 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:22.456787 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213c1ff4-2939-4619-b621-2e156f4af600 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:22.462068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d18c61c-99c7-4420-a809-54a9856f6a21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:22.462734 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18c61c-99c7-4420-a809-54a9856f6a21 service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:22.463020 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18c61c-99c7-4420-a809-54a9856f6a21 service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:22.463296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d18c61c-99c7-4420-a809-54a9856f6a21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:22.472297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18c61c-99c7-4420-a809-54a9856f6a21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:22.473845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d18c61c-99c7-4420-a809-54a9856f6a21 service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:22.483606 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 717/1422] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:23.376657 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae332edc-54bf-44bd-b6dd-8c63af5c427d None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:23.377230 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae332edc-54bf-44bd-b6dd-8c63af5c427d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:23.377230 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae332edc-54bf-44bd-b6dd-8c63af5c427d None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:23.378620 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 22:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:23.383322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b63804-b7ac-4392-9075-5317de945f8d None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:23.383722 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b63804-b7ac-4392-9075-5317de945f8d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:23.383882 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b63804-b7ac-4392-9075-5317de945f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:23.402056 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-d4b63804-b7ac-4392-9075-5317de945f8d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:23.403893 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-d4b63804-b7ac-4392-9075-5317de945f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:23.411514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 22:54:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:23.559219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae332edc-54bf-44bd-b6dd-8c63af5c427d None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:23.582535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae332edc-54bf-44bd-b6dd-8c63af5c427d None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:23.587933 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5d539-6218-40ef-8a24-77a35c2b095a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:23.588576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5d539-6218-40ef-8a24-77a35c2b095a service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:23.588866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5d539-6218-40ef-8a24-77a35c2b095a service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:23.589155 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5d539-6218-40ef-8a24-77a35c2b095a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:23.597736 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5d539-6218-40ef-8a24-77a35c2b095a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:23.599364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5d539-6218-40ef-8a24-77a35c2b095a service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:23.608606 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 707/1425] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:24.501421 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecf342-dcf5-46b7-8171-044bc30d19f9 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:24.501952 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecf342-dcf5-46b7-8171-044bc30d19f9 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:24.501952 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ecf342-dcf5-46b7-8171-044bc30d19f9 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:24.503315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 22:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:24.508051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356e040-ed63-4dac-8cef-b9cf362aba12 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:24.508446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356e040-ed63-4dac-8cef-b9cf362aba12 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:24.508612 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6356e040-ed63-4dac-8cef-b9cf362aba12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:24.526760 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-6356e040-ed63-4dac-8cef-b9cf362aba12 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:24.527981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6356e040-ed63-4dac-8cef-b9cf362aba12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:24.535544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 22:54:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:24.685648 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ecf342-dcf5-46b7-8171-044bc30d19f9 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:24.700274 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ecf342-dcf5-46b7-8171-044bc30d19f9 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:24.705564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a38a4e-53d3-4340-8c7a-9590fff7123e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:24.706197 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a38a4e-53d3-4340-8c7a-9590fff7123e service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:24.706484 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a38a4e-53d3-4340-8c7a-9590fff7123e service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:24.706794 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a38a4e-53d3-4340-8c7a-9590fff7123e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:24.714680 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a38a4e-53d3-4340-8c7a-9590fff7123e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:24.716244 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a38a4e-53d3-4340-8c7a-9590fff7123e service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:24.725501 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 720/1428] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:25.568050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88abf0-3fec-484a-bf35-0ab79c2dcb99 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:25.568571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88abf0-3fec-484a-bf35-0ab79c2dcb99 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:25.568571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88abf0-3fec-484a-bf35-0ab79c2dcb99 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:25.569923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 22:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:25.575493 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f582a5b-bca4-4b33-9e07-bc66cd7d631d None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:25.575916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f582a5b-bca4-4b33-9e07-bc66cd7d631d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:25.576060 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f582a5b-bca4-4b33-9e07-bc66cd7d631d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:25.594294 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-6f582a5b-bca4-4b33-9e07-bc66cd7d631d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:25.595497 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-6f582a5b-bca4-4b33-9e07-bc66cd7d631d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:25.603188 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 22:54:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:25.754307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88abf0-3fec-484a-bf35-0ab79c2dcb99 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:25.769237 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88abf0-3fec-484a-bf35-0ab79c2dcb99 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:25.774502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b08862d-59ce-45a0-adf0-f7b536c109a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:25.775178 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08862d-59ce-45a0-adf0-f7b536c109a4 service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:25.775469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08862d-59ce-45a0-adf0-f7b536c109a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:25.775761 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b08862d-59ce-45a0-adf0-f7b536c109a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:25.784308 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08862d-59ce-45a0-adf0-f7b536c109a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:25.785873 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b08862d-59ce-45a0-adf0-f7b536c109a4 service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:25.795095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 710/1431] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:26.641405 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603935f3-9ed1-448a-b634-1bb257cd4933 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:26.641927 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603935f3-9ed1-448a-b634-1bb257cd4933 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:26.642042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603935f3-9ed1-448a-b634-1bb257cd4933 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:26.643432 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 22:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:26.648253 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594f440-1db1-4959-aafb-2fd76639b447 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:26.648637 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594f440-1db1-4959-aafb-2fd76639b447 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:26.648818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f594f440-1db1-4959-aafb-2fd76639b447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:26.670189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-f594f440-1db1-4959-aafb-2fd76639b447 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:26.671425 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f594f440-1db1-4959-aafb-2fd76639b447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:26.679295 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 22:54:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:26.829199 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603935f3-9ed1-448a-b634-1bb257cd4933 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:26.843725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603935f3-9ed1-448a-b634-1bb257cd4933 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:26.849032 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e539525-e451-4f01-91d7-896d83d96588 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:26.849664 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e539525-e451-4f01-91d7-896d83d96588 service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:26.849951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e539525-e451-4f01-91d7-896d83d96588 service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:26.850230 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e539525-e451-4f01-91d7-896d83d96588 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:26.858777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e539525-e451-4f01-91d7-896d83d96588 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:26.860344 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e539525-e451-4f01-91d7-896d83d96588 service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:26.869460 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 723/1434] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:27.736723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a95345-46b1-48bd-a91a-117299eb4dde None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:27.737280 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a95345-46b1-48bd-a91a-117299eb4dde None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:27.737280 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a95345-46b1-48bd-a91a-117299eb4dde None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:27.738704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 22:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:27.743601 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a2ef5-8206-49ad-a1ec-651279960bf1 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:27.743991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a2ef5-8206-49ad-a1ec-651279960bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:27.744189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a2ef5-8206-49ad-a1ec-651279960bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:27.761721 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-009a2ef5-8206-49ad-a1ec-651279960bf1 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:27.763452 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-009a2ef5-8206-49ad-a1ec-651279960bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:27.771414 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 22:54:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:27.922565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a95345-46b1-48bd-a91a-117299eb4dde None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:27.938076 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a95345-46b1-48bd-a91a-117299eb4dde None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:27.943483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c3fa24-6112-4d06-83c7-04c33f4c0802 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:27.944126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3fa24-6112-4d06-83c7-04c33f4c0802 service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:27.944417 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3fa24-6112-4d06-83c7-04c33f4c0802 service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:27.944700 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c3fa24-6112-4d06-83c7-04c33f4c0802 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:27.952834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3fa24-6112-4d06-83c7-04c33f4c0802 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:27.954406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c3fa24-6112-4d06-83c7-04c33f4c0802 service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:27.964243 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 713/1437] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:28.807761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57272a-2c23-411c-9a2c-03a189a0316b None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:28.808352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57272a-2c23-411c-9a2c-03a189a0316b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:28.808352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57272a-2c23-411c-9a2c-03a189a0316b None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:28.809718 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 22:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:28.814549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ac076-2074-4fbd-aaf4-799f564f829a None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:28.815098 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ac076-2074-4fbd-aaf4-799f564f829a None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:28.815207 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ac076-2074-4fbd-aaf4-799f564f829a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:28.833533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-067ac076-2074-4fbd-aaf4-799f564f829a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:28.834762 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-067ac076-2074-4fbd-aaf4-799f564f829a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:28.843372 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 22:54:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:28.992382 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57272a-2c23-411c-9a2c-03a189a0316b None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:29.007259 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57272a-2c23-411c-9a2c-03a189a0316b None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:29.013119 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36672ad-70e7-4b89-bfdb-0a305cb47b69 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:29.013758 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36672ad-70e7-4b89-bfdb-0a305cb47b69 service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:29.014045 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36672ad-70e7-4b89-bfdb-0a305cb47b69 service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:29.014355 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36672ad-70e7-4b89-bfdb-0a305cb47b69 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:29.022894 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36672ad-70e7-4b89-bfdb-0a305cb47b69 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:29.024459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36672ad-70e7-4b89-bfdb-0a305cb47b69 service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:29.033759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 726/1440] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:31.023707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b928cf-0686-4df3-a47a-cacbb6db20b5 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:31.024287 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b928cf-0686-4df3-a47a-cacbb6db20b5 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:31.024287 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b928cf-0686-4df3-a47a-cacbb6db20b5 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:31.025629 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 22:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:31.030940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5320e97-4b8c-4630-8fe6-a8a3b56c162a None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:31.031332 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5320e97-4b8c-4630-8fe6-a8a3b56c162a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:31.031517 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5320e97-4b8c-4630-8fe6-a8a3b56c162a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:31.049354 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a5320e97-4b8c-4630-8fe6-a8a3b56c162a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:31.050545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a5320e97-4b8c-4630-8fe6-a8a3b56c162a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:31.058340 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 22:54:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:31.096047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b928cf-0686-4df3-a47a-cacbb6db20b5 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:31.107137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b928cf-0686-4df3-a47a-cacbb6db20b5 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:31.110951 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53b4ed0-f683-45de-bfff-fe950cda0746 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:31.111586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b4ed0-f683-45de-bfff-fe950cda0746 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:31.111873 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b4ed0-f683-45de-bfff-fe950cda0746 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:31.112156 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53b4ed0-f683-45de-bfff-fe950cda0746 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:31.120967 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b4ed0-f683-45de-bfff-fe950cda0746 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:31.122860 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53b4ed0-f683-45de-bfff-fe950cda0746 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:31.132096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 716/1443] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:33.392595 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26efb3-1dd8-4a9d-96b6-e1e64b8157f1 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:33.393152 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26efb3-1dd8-4a9d-96b6-e1e64b8157f1 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:33.393152 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a26efb3-1dd8-4a9d-96b6-e1e64b8157f1 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:33.394793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 22:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:33.400050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d860694-ed87-49fd-b937-94a00a1a20b7 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:33.400488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d860694-ed87-49fd-b937-94a00a1a20b7 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:33.400601 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d860694-ed87-49fd-b937-94a00a1a20b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:33.419071 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-4d860694-ed87-49fd-b937-94a00a1a20b7 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:33.420264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-4d860694-ed87-49fd-b937-94a00a1a20b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:33.428031 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 22:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:33.465297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a26efb3-1dd8-4a9d-96b6-e1e64b8157f1 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:33.475987 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a26efb3-1dd8-4a9d-96b6-e1e64b8157f1 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:33.479448 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dc2fa0-0183-42dc-a0cd-91488887ec3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:33.480668 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc2fa0-0183-42dc-a0cd-91488887ec3b service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:33.480961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc2fa0-0183-42dc-a0cd-91488887ec3b service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:33.481235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dc2fa0-0183-42dc-a0cd-91488887ec3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:33.490372 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc2fa0-0183-42dc-a0cd-91488887ec3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:33.491796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dc2fa0-0183-42dc-a0cd-91488887ec3b service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:33.500991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 729/1446] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:35.914016 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2a7b3-97b4-4346-aced-4d9e19cfd8a2 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:35.914542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2a7b3-97b4-4346-aced-4d9e19cfd8a2 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:35.914679 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a2a7b3-97b4-4346-aced-4d9e19cfd8a2 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:35.915986 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 22:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:35.921616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcc93b-0d37-4232-8e6e-9050faab5a95 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:35.922041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcc93b-0d37-4232-8e6e-9050faab5a95 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:35.922263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfcc93b-0d37-4232-8e6e-9050faab5a95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:35.939430 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-ecfcc93b-0d37-4232-8e6e-9050faab5a95 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:35.940604 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ecfcc93b-0d37-4232-8e6e-9050faab5a95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:35.948154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 22:54:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:35.989337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2a7b3-97b4-4346-aced-4d9e19cfd8a2 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:35.999863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a2a7b3-97b4-4346-aced-4d9e19cfd8a2 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:36.003888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd3209b-7f56-46e8-b88d-821e16e4d93f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:36.004556 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3209b-7f56-46e8-b88d-821e16e4d93f service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:36.004873 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3209b-7f56-46e8-b88d-821e16e4d93f service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:36.005193 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd3209b-7f56-46e8-b88d-821e16e4d93f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:36.014843 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd3209b-7f56-46e8-b88d-821e16e4d93f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:36.016268 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd3209b-7f56-46e8-b88d-821e16e4d93f service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:36.025461 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 719/1449] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:38.251924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a394afd-186f-4517-9c1b-dda026459e1b None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:38.252440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a394afd-186f-4517-9c1b-dda026459e1b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:38.252547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a394afd-186f-4517-9c1b-dda026459e1b None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:38.253871 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 22:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:38.258969 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d73e6-3240-4ae9-81c5-2fef3672bed0 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:38.259389 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d73e6-3240-4ae9-81c5-2fef3672bed0 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:38.259510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76d73e6-3240-4ae9-81c5-2fef3672bed0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:38.277175 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-e76d73e6-3240-4ae9-81c5-2fef3672bed0 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:38.278391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-e76d73e6-3240-4ae9-81c5-2fef3672bed0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:38.286118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 22:54:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:38.324630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a394afd-186f-4517-9c1b-dda026459e1b None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:38.326319 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-6a394afd-186f-4517-9c1b-dda026459e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:38.340072 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a394afd-186f-4517-9c1b-dda026459e1b None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:38.343600 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e562320-6993-4d84-9c6a-a87dc671ed6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:38.344243 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e562320-6993-4d84-9c6a-a87dc671ed6f service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:38.344527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e562320-6993-4d84-9c6a-a87dc671ed6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:38.344806 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e562320-6993-4d84-9c6a-a87dc671ed6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:38.353704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e562320-6993-4d84-9c6a-a87dc671ed6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:38.355654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e562320-6993-4d84-9c6a-a87dc671ed6f service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:38.364843 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 732/1452] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:40.595707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd5785-30f4-4842-ba6e-327ad15a7934 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:40.596208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd5785-30f4-4842-ba6e-327ad15a7934 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:40.596208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bd5785-30f4-4842-ba6e-327ad15a7934 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:40.597562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 22:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:40.602817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a3451-b7a4-4756-9160-b48e3b5bc8ee None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:40.603206 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a3451-b7a4-4756-9160-b48e3b5bc8ee None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:40.603391 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5a3451-b7a4-4756-9160-b48e3b5bc8ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:40.621437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-1b5a3451-b7a4-4756-9160-b48e3b5bc8ee None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:40.622662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-1b5a3451-b7a4-4756-9160-b48e3b5bc8ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:40.630375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 22:54:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:40.668223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd5785-30f4-4842-ba6e-327ad15a7934 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:40.678803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bd5785-30f4-4842-ba6e-327ad15a7934 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:40.682228 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a87ff-247e-4829-bdba-eae5f48d4709 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:40.682904 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a87ff-247e-4829-bdba-eae5f48d4709 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:40.683195 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a87ff-247e-4829-bdba-eae5f48d4709 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:40.683475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a87ff-247e-4829-bdba-eae5f48d4709 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:40.692084 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a87ff-247e-4829-bdba-eae5f48d4709 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:40.693499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a87ff-247e-4829-bdba-eae5f48d4709 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:40.703132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 722/1455] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:42.880816 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a883949-9659-44eb-b277-56708af4b65a None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:42.881356 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a883949-9659-44eb-b277-56708af4b65a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:42.881356 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a883949-9659-44eb-b277-56708af4b65a None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:42.882780 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 22:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:54:42.887940 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b915c-c574-4f36-b07a-531169a50472 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:42.888333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b915c-c574-4f36-b07a-531169a50472 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:42.888510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b915c-c574-4f36-b07a-531169a50472 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:42.906240 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-563b915c-c574-4f36-b07a-531169a50472 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:42.907465 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-563b915c-c574-4f36-b07a-531169a50472 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:42.915349 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 22:54:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:42.958748 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a883949-9659-44eb-b277-56708af4b65a None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:42.969317 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a883949-9659-44eb-b277-56708af4b65a None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:42.972829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5b2f2-71d7-494f-818f-b07d8ab529b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:42.973465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5b2f2-71d7-494f-818f-b07d8ab529b7 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:42.973752 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5b2f2-71d7-494f-818f-b07d8ab529b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:42.974034 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5b2f2-71d7-494f-818f-b07d8ab529b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:42.983228 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5b2f2-71d7-494f-818f-b07d8ab529b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:42.984637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5b2f2-71d7-494f-818f-b07d8ab529b7 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:42.993691 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 735/1458] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:45.160113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeec62c-5a58-42d6-a2f5-6a99cb6f650a None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:45.160681 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeec62c-5a58-42d6-a2f5-6a99cb6f650a None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:45.160681 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eeec62c-5a58-42d6-a2f5-6a99cb6f650a None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:45.162085 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 22:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:45.167833 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fac3b8-1c90-436a-81f2-7f6fed212538 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:45.168253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fac3b8-1c90-436a-81f2-7f6fed212538 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:45.168392 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fac3b8-1c90-436a-81f2-7f6fed212538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:45.186118 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-55fac3b8-1c90-436a-81f2-7f6fed212538 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:45.187338 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-55fac3b8-1c90-436a-81f2-7f6fed212538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:45.195052 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 22:54:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:45.231712 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeec62c-5a58-42d6-a2f5-6a99cb6f650a None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:45.242474 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eeec62c-5a58-42d6-a2f5-6a99cb6f650a None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:45.247024 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a36d7-4741-4393-b081-dd6ad712b07c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:45.247699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a36d7-4741-4393-b081-dd6ad712b07c service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:45.247992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a36d7-4741-4393-b081-dd6ad712b07c service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:45.248273 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a36d7-4741-4393-b081-dd6ad712b07c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:45.257216 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a36d7-4741-4393-b081-dd6ad712b07c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:45.258661 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a36d7-4741-4393-b081-dd6ad712b07c service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:45.268182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 725/1461] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:47.421710 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9f290-2931-471e-838a-4466ae123699 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:47.422261 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9f290-2931-471e-838a-4466ae123699 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:47.422261 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d9f290-2931-471e-838a-4466ae123699 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:47.423691 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 22:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:47.428987 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69754fc1-5745-4c2c-8702-9856a67dcc48 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:47.429408 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69754fc1-5745-4c2c-8702-9856a67dcc48 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:47.429548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69754fc1-5745-4c2c-8702-9856a67dcc48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:47.436215 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=79 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:54:47.448560 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-69754fc1-5745-4c2c-8702-9856a67dcc48 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:47.449771 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-69754fc1-5745-4c2c-8702-9856a67dcc48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:47.457455 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 22:54:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:47.494048 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9f290-2931-471e-838a-4466ae123699 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:47.499958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=73 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:54:47.506377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d9f290-2931-471e-838a-4466ae123699 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:47.509842 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fabec2e-f1ea-455c-9f53-c6368f1afbe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:47.510522 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabec2e-f1ea-455c-9f53-c6368f1afbe0 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:47.510836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabec2e-f1ea-455c-9f53-c6368f1afbe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:47.511111 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fabec2e-f1ea-455c-9f53-c6368f1afbe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:47.519998 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fabec2e-f1ea-455c-9f53-c6368f1afbe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:47.521399 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fabec2e-f1ea-455c-9f53-c6368f1afbe0 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:47.530605 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 738/1464] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:49.643075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23050f-b258-4b68-95a4-ae0f766a7ea4 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:49.643605 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23050f-b258-4b68-95a4-ae0f766a7ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:49.643723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d23050f-b258-4b68-95a4-ae0f766a7ea4 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:49.645088 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 22:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:49.651061 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08508d-cc49-4a6f-8197-65080f751876 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:49.651488 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08508d-cc49-4a6f-8197-65080f751876 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:49.651628 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a08508d-cc49-4a6f-8197-65080f751876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:49.670835 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-3a08508d-cc49-4a6f-8197-65080f751876 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:49.672032 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-3a08508d-cc49-4a6f-8197-65080f751876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:49.679886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 22:54:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:49.716351 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23050f-b258-4b68-95a4-ae0f766a7ea4 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:49.727083 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d23050f-b258-4b68-95a4-ae0f766a7ea4 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:49.730574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d12e4-fef3-468c-99cf-80b76e7da898 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:49.731238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d12e4-fef3-468c-99cf-80b76e7da898 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:49.731528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d12e4-fef3-468c-99cf-80b76e7da898 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:49.731809 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d12e4-fef3-468c-99cf-80b76e7da898 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:49.740635 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d12e4-fef3-468c-99cf-80b76e7da898 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:49.742054 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d12e4-fef3-468c-99cf-80b76e7da898 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:49.751331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 728/1467] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:54:51.883294 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d1759-2020-4143-94a7-e6360eb1bd3b None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:51.883850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d1759-2020-4143-94a7-e6360eb1bd3b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:51.883850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4d1759-2020-4143-94a7-e6360eb1bd3b None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:51.885734 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 22:54:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:54:51.891300 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab924366-ff94-42c6-ad82-6c46a448b90a None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:51.891695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab924366-ff94-42c6-ad82-6c46a448b90a None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:51.891873 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab924366-ff94-42c6-ad82-6c46a448b90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:51.911162 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-ab924366-ff94-42c6-ad82-6c46a448b90a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:54:51.912366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-ab924366-ff94-42c6-ad82-6c46a448b90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:54:51.922078 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 22:54:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:54:51.958369 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d1759-2020-4143-94a7-e6360eb1bd3b None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:54:51.969720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4d1759-2020-4143-94a7-e6360eb1bd3b None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:54:51.973291 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca4569f-4b64-4e46-9a73-6088e6aeea54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:54:51.973931 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4569f-4b64-4e46-9a73-6088e6aeea54 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:54:51.974214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4569f-4b64-4e46-9a73-6088e6aeea54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:54:51.974490 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca4569f-4b64-4e46-9a73-6088e6aeea54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:54:51.983353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca4569f-4b64-4e46-9a73-6088e6aeea54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:54:51.984774 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca4569f-4b64-4e46-9a73-6088e6aeea54 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:54:51.994910 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 741/1470] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:54:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:55:01.905391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=11 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:55:01.990069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=14 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:55:11.864710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354ab9c-32b1-4c22-92b4-b668255bed59 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:11.865284 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354ab9c-32b1-4c22-92b4-b668255bed59 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:11.865284 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f354ab9c-32b1-4c22-92b4-b668255bed59 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:11.866659 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 730/1471] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:55:11.872242 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6835a1f-02d9-4e05-b01a-801a8d9d7375 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:11.872632 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6835a1f-02d9-4e05-b01a-801a8d9d7375 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:11.872807 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6835a1f-02d9-4e05-b01a-801a8d9d7375 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:11.891821 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a6835a1f-02d9-4e05-b01a-801a8d9d7375 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:55:11.893026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a6835a1f-02d9-4e05-b01a-801a8d9d7375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:55:11.901071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 742/1472] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:55:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:55:11.948072 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d094b1-edaa-4022-9908-cf97f55bba66 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:11.948366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d094b1-edaa-4022-9908-cf97f55bba66 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:11.948646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d094b1-edaa-4022-9908-cf97f55bba66 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:11.950009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 731/1473] 10.222.0.17 () {54 vars in 962 bytes} [Thu Jun 16 22:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:55:11.954825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2edfde-b838-4b9e-acee-cbdecea35f81 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:11.955118 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2edfde-b838-4b9e-acee-cbdecea35f81 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:11.955401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2edfde-b838-4b9e-acee-cbdecea35f81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:11.973220 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-ca2edfde-b838-4b9e-acee-cbdecea35f81 None None] MFA Rules not processed for user `915f4296504b47d796e3a5b99550eb75`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:55:11.974402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ca2edfde-b838-4b9e-acee-cbdecea35f81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:55:11.981536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 743/1474] 10.222.0.17 () {58 vars in 1059 bytes} [Thu Jun 16 22:55:11 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:55:11.988490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d094b1-edaa-4022-9908-cf97f55bba66 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:55:11.999348 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d094b1-edaa-4022-9908-cf97f55bba66 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:55:12.002781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7654460-cf82-406e-a524-9d651a95b291 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915f4296504b47d796e3a5b99550eb75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:55:12.003397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7654460-cf82-406e-a524-9d651a95b291 service glance] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:12.003712 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7654460-cf82-406e-a524-9d651a95b291 service glance] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:12.003959 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7654460-cf82-406e-a524-9d651a95b291 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:12.014096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7654460-cf82-406e-a524-9d651a95b291 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:55:12.015549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7654460-cf82-406e-a524-9d651a95b291 service glance] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:55:12.024676 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 732/1475] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:55:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:55:14.217607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bfa50-ef12-4ff7-8563-69e1bbd6d23c None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:14.218121 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bfa50-ef12-4ff7-8563-69e1bbd6d23c None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:14.218121 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6bfa50-ef12-4ff7-8563-69e1bbd6d23c None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:14.219509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 744/1476] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:55:14.224796 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767d69c-41e4-4706-95be-8dfee5003674 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:14.225226 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767d69c-41e4-4706-95be-8dfee5003674 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:14.225357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6767d69c-41e4-4706-95be-8dfee5003674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:14.243460 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-6767d69c-41e4-4706-95be-8dfee5003674 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:55:14.244657 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6767d69c-41e4-4706-95be-8dfee5003674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:55:14.252396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 733/1477] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:55:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:55:14.276818 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6bfa50-ef12-4ff7-8563-69e1bbd6d23c None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:55:14.289351 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6bfa50-ef12-4ff7-8563-69e1bbd6d23c None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:55:14.293805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae62587-0e94-411d-910c-d121f42566f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:55:14.294450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae62587-0e94-411d-910c-d121f42566f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:14.294774 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae62587-0e94-411d-910c-d121f42566f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:14.295060 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae62587-0e94-411d-910c-d121f42566f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:14.295791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae62587-0e94-411d-910c-d121f42566f3 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:55:14.297570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: WARNING py.warnings [None req-7ae62587-0e94-411d-910c-d121f42566f3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 22:55:14.297570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: warnings.warn(msg) Jun 16 22:55:14.297570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: Jun 16 22:55:14.298757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae62587-0e94-411d-910c-d121f42566f3 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:55:14.317624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 745/1478] 10.222.0.17 () {56 vars in 1108 bytes} [Thu Jun 16 22:55:14 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 22:55:16.438784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2acfd-b1cc-4ba3-aff5-3356097c7583 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:16.439353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2acfd-b1cc-4ba3-aff5-3356097c7583 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:16.439353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f2acfd-b1cc-4ba3-aff5-3356097c7583 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:16.440723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 734/1479] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:55:16.446500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c2f1c-26e8-4439-8e12-03cfeb5fb87b None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:16.446941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c2f1c-26e8-4439-8e12-03cfeb5fb87b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:16.447103 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494c2f1c-26e8-4439-8e12-03cfeb5fb87b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:16.465005 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-494c2f1c-26e8-4439-8e12-03cfeb5fb87b None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:55:16.466214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-494c2f1c-26e8-4439-8e12-03cfeb5fb87b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:55:16.474049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 746/1480] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:55:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:55:16.516966 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2acfd-b1cc-4ba3-aff5-3356097c7583 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:55:16.531468 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f2acfd-b1cc-4ba3-aff5-3356097c7583 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:55:16.536848 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc8195-e88c-4bf9-888f-cdc136d63692 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:55:16.537486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc8195-e88c-4bf9-888f-cdc136d63692 service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:16.537778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc8195-e88c-4bf9-888f-cdc136d63692 service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:16.538063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc8195-e88c-4bf9-888f-cdc136d63692 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:16.546603 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc8195-e88c-4bf9-888f-cdc136d63692 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:55:16.548304 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc8195-e88c-4bf9-888f-cdc136d63692 service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:55:16.558265 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 735/1481] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:55:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 22:55:18.866849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055f069-a9be-44ce-8906-99e3e13e956c None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:18.867411 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055f069-a9be-44ce-8906-99e3e13e956c None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:18.867411 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a055f069-a9be-44ce-8906-99e3e13e956c None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:18.868856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 747/1482] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:55:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:55:18.874082 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5ccfe-dc4a-45e8-8bde-4e5b4a686106 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:18.874511 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5ccfe-dc4a-45e8-8bde-4e5b4a686106 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:18.874665 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb5ccfe-dc4a-45e8-8bde-4e5b4a686106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:18.892488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-3eb5ccfe-dc4a-45e8-8bde-4e5b4a686106 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:55:18.893686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-3eb5ccfe-dc4a-45e8-8bde-4e5b4a686106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:55:18.901411 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 736/1483] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:55:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:55:18.937415 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055f069-a9be-44ce-8906-99e3e13e956c None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:55:18.947909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a055f069-a9be-44ce-8906-99e3e13e956c None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:55:18.951598 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d734c38-9012-4627-b351-9b23f240484e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:55:18.952246 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d734c38-9012-4627-b351-9b23f240484e service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:18.952565 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d734c38-9012-4627-b351-9b23f240484e service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:18.952845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d734c38-9012-4627-b351-9b23f240484e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:18.961721 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d734c38-9012-4627-b351-9b23f240484e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:55:18.963180 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d734c38-9012-4627-b351-9b23f240484e service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:55:18.972367 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 748/1484] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:55:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:55:21.130744 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407f18a-8863-4541-acc0-58a33762d9ad None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:21.131282 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407f18a-8863-4541-acc0-58a33762d9ad None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:21.131282 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3407f18a-8863-4541-acc0-58a33762d9ad None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:21.132793 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 737/1485] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:55:21.138373 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31faa8-10db-49eb-9b17-250130ff8483 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:21.138834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31faa8-10db-49eb-9b17-250130ff8483 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:21.138965 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa31faa8-10db-49eb-9b17-250130ff8483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:21.156137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-fa31faa8-10db-49eb-9b17-250130ff8483 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:55:21.157316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-fa31faa8-10db-49eb-9b17-250130ff8483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:55:21.165503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 749/1486] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:55:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:55:21.203707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407f18a-8863-4541-acc0-58a33762d9ad None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:55:21.214502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3407f18a-8863-4541-acc0-58a33762d9ad None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:55:21.218013 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a76f447-02e0-44e0-8b00-06f2df1fd38b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:55:21.218680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76f447-02e0-44e0-8b00-06f2df1fd38b service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:21.218967 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76f447-02e0-44e0-8b00-06f2df1fd38b service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:21.219243 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a76f447-02e0-44e0-8b00-06f2df1fd38b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:21.228571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76f447-02e0-44e0-8b00-06f2df1fd38b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:55:21.229986 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a76f447-02e0-44e0-8b00-06f2df1fd38b service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:55:21.239338 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 738/1487] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:55:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:55:23.475537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b913070-1149-4104-a638-7267a0bd730c None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:23.476102 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b913070-1149-4104-a638-7267a0bd730c None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:23.476102 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b913070-1149-4104-a638-7267a0bd730c None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:23.477457 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 750/1488] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:55:23.482584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc09302-95b1-441b-85a6-086c506952a9 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:23.483139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc09302-95b1-441b-85a6-086c506952a9 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:23.483247 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc09302-95b1-441b-85a6-086c506952a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:23.500917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-bbc09302-95b1-441b-85a6-086c506952a9 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:55:23.502043 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-bbc09302-95b1-441b-85a6-086c506952a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:55:23.510506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 739/1489] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:55:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:55:23.550304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b913070-1149-4104-a638-7267a0bd730c None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:55:23.560821 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b913070-1149-4104-a638-7267a0bd730c None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:55:23.564251 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a1671-d055-4aa5-b754-a61bbc7edfce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:55:23.564888 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a1671-d055-4aa5-b754-a61bbc7edfce service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:23.565173 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a1671-d055-4aa5-b754-a61bbc7edfce service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:23.565450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a1671-d055-4aa5-b754-a61bbc7edfce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:23.574565 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a1671-d055-4aa5-b754-a61bbc7edfce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:55:23.576004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a1671-d055-4aa5-b754-a61bbc7edfce service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:55:23.585726 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 751/1490] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:55:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:55:25.856936 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb6b9a-f76a-4dff-aacd-708a8b1cd165 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:25.857511 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb6b9a-f76a-4dff-aacd-708a8b1cd165 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:25.857511 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bb6b9a-f76a-4dff-aacd-708a8b1cd165 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:25.858951 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 740/1491] 10.222.0.17 () {54 vars in 926 bytes} [Thu Jun 16 22:55:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:55:25.863978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4443ac4-5ed5-4b2a-a5d7-84e93b7d51c1 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:25.864410 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4443ac4-5ed5-4b2a-a5d7-84e93b7d51c1 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:25.864548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4443ac4-5ed5-4b2a-a5d7-84e93b7d51c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:25.883050 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-e4443ac4-5ed5-4b2a-a5d7-84e93b7d51c1 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:55:25.884250 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e4443ac4-5ed5-4b2a-a5d7-84e93b7d51c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:55:25.892314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 752/1492] 10.222.0.17 () {58 vars in 1023 bytes} [Thu Jun 16 22:55:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:55:25.930608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb6b9a-f76a-4dff-aacd-708a8b1cd165 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:55:25.941086 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bb6b9a-f76a-4dff-aacd-708a8b1cd165 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:55:25.945080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b5218-550b-46b7-ba80-dec042264a54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:55:25.945711 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b5218-550b-46b7-ba80-dec042264a54 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:55:25.946376 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b5218-550b-46b7-ba80-dec042264a54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:55:25.946376 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b5218-550b-46b7-ba80-dec042264a54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:55:25.955289 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b5218-550b-46b7-ba80-dec042264a54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:55:25.956721 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102b5218-550b-46b7-ba80-dec042264a54 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:55:25.966933 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 741/1493] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:55:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:55:35.877350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=33 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:55:35.962633 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=33 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:56:51.573461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef4bdb-f62f-4d68-b5c8-da376bade6c2 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:56:51.574015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef4bdb-f62f-4d68-b5c8-da376bade6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:56:51.574015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ef4bdb-f62f-4d68-b5c8-da376bade6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:56:51.592355 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-c2ef4bdb-f62f-4d68-b5c8-da376bade6c2 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:56:51.593572 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c2ef4bdb-f62f-4d68-b5c8-da376bade6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:56:51.601468 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 753/1494] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 22:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:56:51.610460 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516387af-2d6f-4f84-881e-689d0e5730dc None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:56:51.611008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516387af-2d6f-4f84-881e-689d0e5730dc None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:56:51.611113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516387af-2d6f-4f84-881e-689d0e5730dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:56:51.629136 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-516387af-2d6f-4f84-881e-689d0e5730dc None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:56:51.630580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-516387af-2d6f-4f84-881e-689d0e5730dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:56:51.638896 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 742/1495] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 22:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:56:51.643210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef4bdb-f62f-4d68-b5c8-da376bade6c2 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:56:51.652494 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ef4bdb-f62f-4d68-b5c8-da376bade6c2 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:56:51.657899 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5b9bb-576c-41e4-99ca-84229e0e9332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:56:51.658546 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5b9bb-576c-41e4-99ca-84229e0e9332 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:56:51.658864 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5b9bb-576c-41e4-99ca-84229e0e9332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:56:51.659143 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e5b9bb-576c-41e4-99ca-84229e0e9332 admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:56:51.659853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5b9bb-576c-41e4-99ca-84229e0e9332 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 22:56:51.660161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5b9bb-576c-41e4-99ca-84229e0e9332 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:56:51.661161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: WARNING py.warnings [None req-13e5b9bb-576c-41e4-99ca-84229e0e9332 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 22:56:51.661161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: warnings.warn(msg) Jun 16 22:56:51.661161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: Jun 16 22:56:51.662029 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e5b9bb-576c-41e4-99ca-84229e0e9332 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:56:51.673837 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 754/1496] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 22:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:56:51.681567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516387af-2d6f-4f84-881e-689d0e5730dc None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:56:51.690542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516387af-2d6f-4f84-881e-689d0e5730dc None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:56:51.695042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b75459-4f48-4c24-8027-31b1c2e8a66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:56:51.695683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b75459-4f48-4c24-8027-31b1c2e8a66d admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:56:51.695968 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b75459-4f48-4c24-8027-31b1c2e8a66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:56:51.696247 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b75459-4f48-4c24-8027-31b1c2e8a66d admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:56:51.696941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b75459-4f48-4c24-8027-31b1c2e8a66d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:56:51.698713 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: WARNING py.warnings [None req-b6b75459-4f48-4c24-8027-31b1c2e8a66d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 22:56:51.698713 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: warnings.warn(msg) Jun 16 22:56:51.698713 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: Jun 16 22:56:51.699929 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b75459-4f48-4c24-8027-31b1c2e8a66d admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:56:51.742164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 743/1497] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 22:56:51 2022] POST /identity/v3/projects => generated 377 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 22:56:51.746099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5b9bb-576c-41e4-99ca-84229e0e9332 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:56:51.756249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e5b9bb-576c-41e4-99ca-84229e0e9332 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:56:51.811409 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02dfe7-b42b-4c9d-a79e-aa579eaf0bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:56:51.812067 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02dfe7-b42b-4c9d-a79e-aa579eaf0bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:56:51.812355 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02dfe7-b42b-4c9d-a79e-aa579eaf0bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:56:51.812635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02dfe7-b42b-4c9d-a79e-aa579eaf0bb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:56:51.813328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02dfe7-b42b-4c9d-a79e-aa579eaf0bb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:56:51.814334 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: WARNING py.warnings [None req-1a02dfe7-b42b-4c9d-a79e-aa579eaf0bb6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 22:56:51.814334 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: warnings.warn(msg) Jun 16 22:56:51.814334 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: Jun 16 22:56:51.815571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02dfe7-b42b-4c9d-a79e-aa579eaf0bb6 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:56:51.835022 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 755/1498] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 22:56:51 2022] POST /identity/v3/users => generated 289 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 22:56:51.838991 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b75459-4f48-4c24-8027-31b1c2e8a66d admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:56:51.851259 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b75459-4f48-4c24-8027-31b1c2e8a66d admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:56:51.855716 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a8af4-75c3-4a4b-905e-62aea7af96ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:56:51.856430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a8af4-75c3-4a4b-905e-62aea7af96ce admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:56:51.856713 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a8af4-75c3-4a4b-905e-62aea7af96ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:56:51.856985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a8af4-75c3-4a4b-905e-62aea7af96ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:56:51.857696 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a8af4-75c3-4a4b-905e-62aea7af96ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:56:51.858728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: WARNING py.warnings [None req-420a8af4-75c3-4a4b-905e-62aea7af96ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 22:56:51.858728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: warnings.warn(msg) Jun 16 22:56:51.858728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: Jun 16 22:56:51.859594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a8af4-75c3-4a4b-905e-62aea7af96ce admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:56:51.872339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 744/1499] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 22:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 22:56:51.876291 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02dfe7-b42b-4c9d-a79e-aa579eaf0bb6 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:56:51.886221 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02dfe7-b42b-4c9d-a79e-aa579eaf0bb6 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:56:51.891257 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8ef3d-4bb5-4055-9a8b-18a8d804538b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:56:51.891912 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8ef3d-4bb5-4055-9a8b-18a8d804538b admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:56:51.892200 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8ef3d-4bb5-4055-9a8b-18a8d804538b admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:56:51.892480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8ef3d-4bb5-4055-9a8b-18a8d804538b admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:56:51.893205 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8ef3d-4bb5-4055-9a8b-18a8d804538b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:56:51.894202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: WARNING py.warnings [None req-a6a8ef3d-4bb5-4055-9a8b-18a8d804538b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 22:56:51.894202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: warnings.warn(msg) Jun 16 22:56:51.894202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: Jun 16 22:56:51.895128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8ef3d-4bb5-4055-9a8b-18a8d804538b admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:56:51.909051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 756/1500] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 22:56:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 22:56:51.912681 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a8af4-75c3-4a4b-905e-62aea7af96ce admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:56:51.923079 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a8af4-75c3-4a4b-905e-62aea7af96ce admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:56:51.927493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f3235a-3c2b-486a-9f61-30a2b8d1e442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:56:51.928363 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3235a-3c2b-486a-9f61-30a2b8d1e442 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:56:51.928652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3235a-3c2b-486a-9f61-30a2b8d1e442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:56:51.928933 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f3235a-3c2b-486a-9f61-30a2b8d1e442 admin admin] PATH_INFO: `/v3/projects/eafb4e17294741e68b70cf9c5e893760/users/fb634482fc6041c7a371a4e094741f22/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:56:51.943724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3235a-3c2b-486a-9f61-30a2b8d1e442 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eafb4e17294741e68b70cf9c5e893760, user_id=fb634482fc6041c7a371a4e094741f22, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:56:51.944732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: WARNING py.warnings [None req-c4f3235a-3c2b-486a-9f61-30a2b8d1e442 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 22:56:51.944732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: warnings.warn(msg) Jun 16 22:56:51.944732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: Jun 16 22:56:51.946651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f3235a-3c2b-486a-9f61-30a2b8d1e442 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:56:51.958914 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 745/1501] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 22:56:51 2022] PUT /identity/v3/projects/eafb4e17294741e68b70cf9c5e893760/users/fb634482fc6041c7a371a4e094741f22/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:56:51.969988 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9a589-b362-42c0-8711-a9cba054b31a None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:56:51.970281 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9a589-b362-42c0-8711-a9cba054b31a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:56:51.970562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f9a589-b362-42c0-8711-a9cba054b31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:56:51.997715 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-c5f9a589-b362-42c0-8711-a9cba054b31a None None] MFA Rules not processed for user `fb634482fc6041c7a371a4e094741f22`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:56:51.998950 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c5f9a589-b362-42c0-8711-a9cba054b31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:56:52.063241 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 757/1502] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 22:56:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:56:52.076431 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac34529-3d26-4a63-b3b2-9d481a8715a4 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:56:52.076724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac34529-3d26-4a63-b3b2-9d481a8715a4 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:56:52.077007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac34529-3d26-4a63-b3b2-9d481a8715a4 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:56:52.078367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 746/1503] 10.222.0.17 () {54 vars in 963 bytes} [Thu Jun 16 22:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 22:56:52.083217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf57b6a-c1d3-44f4-b238-d7b0369f0611 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:56:52.083515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf57b6a-c1d3-44f4-b238-d7b0369f0611 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:56:52.083801 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf57b6a-c1d3-44f4-b238-d7b0369f0611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:56:52.101427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-ddf57b6a-c1d3-44f4-b238-d7b0369f0611 None None] MFA Rules not processed for user `24ca7f221334442781f10574277bf422`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:56:52.102567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ddf57b6a-c1d3-44f4-b238-d7b0369f0611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:56:52.147290 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:56:52.156243 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 758/1504] 10.222.0.17 () {58 vars in 1060 bytes} [Thu Jun 16 22:56:52 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:56:52.163782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac34529-3d26-4a63-b3b2-9d481a8715a4 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:56:52.174610 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac34529-3d26-4a63-b3b2-9d481a8715a4 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:56:52.178064 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61d4ac-ebcd-4147-8353-2708a143b66b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:56:52.178725 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61d4ac-ebcd-4147-8353-2708a143b66b service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:56:52.179009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61d4ac-ebcd-4147-8353-2708a143b66b service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:56:52.179288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c61d4ac-ebcd-4147-8353-2708a143b66b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:56:52.187852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61d4ac-ebcd-4147-8353-2708a143b66b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:56:52.189258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c61d4ac-ebcd-4147-8353-2708a143b66b service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:56:52.197776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 747/1505] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:56:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:56:52.240081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750cb7b-6aeb-4364-82f9-dac0a32f53cf None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:56:52.240369 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750cb7b-6aeb-4364-82f9-dac0a32f53cf None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:56:52.240673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5750cb7b-6aeb-4364-82f9-dac0a32f53cf None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:56:52.241984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 759/1506] 10.222.0.17 () {52 vars in 829 bytes} [Thu Jun 16 22:56:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:56:52.245571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61d4ac-ebcd-4147-8353-2708a143b66b service cinder] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:56:52.307223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c61d4ac-ebcd-4147-8353-2708a143b66b service cinder] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:56:52.311754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5916d-bf4f-48b8-bce0-f1abf441a7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:56:52.312627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5916d-bf4f-48b8-bce0-f1abf441a7c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:56:52.312919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5916d-bf4f-48b8-bce0-f1abf441a7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:56:52.313201 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a5916d-bf4f-48b8-bce0-f1abf441a7c6 admin admin] PATH_INFO: `/v3/users/fb634482fc6041c7a371a4e094741f22` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:56:52.314384 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5916d-bf4f-48b8-bce0-f1abf441a7c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb634482fc6041c7a371a4e094741f22)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:56:52.315445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: WARNING py.warnings [None req-c4a5916d-bf4f-48b8-bce0-f1abf441a7c6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 22:56:52.315445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: warnings.warn(msg) Jun 16 22:56:52.315445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: Jun 16 22:56:52.316663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a5916d-bf4f-48b8-bce0-f1abf441a7c6 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:56:52.381932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c4a5916d-bf4f-48b8-bce0-f1abf441a7c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb634482fc6041c7a371a4e094741f22'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 22:56:52.382648 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c4a5916d-bf4f-48b8-bce0-f1abf441a7c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb634482fc6041c7a371a4e094741f22'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 22:56:52.395272 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c4a5916d-bf4f-48b8-bce0-f1abf441a7c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb634482fc6041c7a371a4e094741f22'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 22:56:52.396063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c4a5916d-bf4f-48b8-bce0-f1abf441a7c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb634482fc6041c7a371a4e094741f22'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 22:56:52.414532 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c4a5916d-bf4f-48b8-bce0-f1abf441a7c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb634482fc6041c7a371a4e094741f22'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 22:56:52.418448 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:56:52.431428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 748/1507] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 22:56:52 2022] DELETE /identity/v3/users/fb634482fc6041c7a371a4e094741f22 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:56:52.436048 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7c5db-2ec6-4301-be99-c5e0da7044e2 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:56:52.436346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7c5db-2ec6-4301-be99-c5e0da7044e2 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:56:52.436630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b7c5db-2ec6-4301-be99-c5e0da7044e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:56:52.453544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-f5b7c5db-2ec6-4301-be99-c5e0da7044e2 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:56:52.454796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-f5b7c5db-2ec6-4301-be99-c5e0da7044e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:56:52.512421 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 760/1508] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 22:56:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:56:52.523917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5916d-bf4f-48b8-bce0-f1abf441a7c6 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:56:52.525345 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c4a5916d-bf4f-48b8-bce0-f1abf441a7c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:56:52.584314 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a5916d-bf4f-48b8-bce0-f1abf441a7c6 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:56:52.587924 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca5eaa5-1050-49ee-9ec0-880410ecbd7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:56:52.588574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5eaa5-1050-49ee-9ec0-880410ecbd7b service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:56:52.588860 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5eaa5-1050-49ee-9ec0-880410ecbd7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:56:52.589141 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca5eaa5-1050-49ee-9ec0-880410ecbd7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:56:52.598549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca5eaa5-1050-49ee-9ec0-880410ecbd7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:56:52.600008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca5eaa5-1050-49ee-9ec0-880410ecbd7b service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:56:52.609328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 749/1509] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:56:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:56:52.937641 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7c5db-2ec6-4301-be99-c5e0da7044e2 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:56:52.939226 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-f5b7c5db-2ec6-4301-be99-c5e0da7044e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:56:52.951817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b7c5db-2ec6-4301-be99-c5e0da7044e2 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:56:52.957269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73860959-cdda-4424-9d65-fcda9031ef80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:56:52.958142 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73860959-cdda-4424-9d65-fcda9031ef80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:56:52.958426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73860959-cdda-4424-9d65-fcda9031ef80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:56:52.958891 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73860959-cdda-4424-9d65-fcda9031ef80 admin admin] PATH_INFO: `/v3/projects/eafb4e17294741e68b70cf9c5e893760` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:56:52.966623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73860959-cdda-4424-9d65-fcda9031ef80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eafb4e17294741e68b70cf9c5e893760)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:56:52.966623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: WARNING py.warnings [None req-73860959-cdda-4424-9d65-fcda9031ef80 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 22:56:52.966623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: warnings.warn(msg) Jun 16 22:56:52.966623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: Jun 16 22:56:52.966623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73860959-cdda-4424-9d65-fcda9031ef80 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:56:53.023172 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-73860959-cdda-4424-9d65-fcda9031ef80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eafb4e17294741e68b70cf9c5e893760', 'request_id': 'req-73860959-cdda-4424-9d65-fcda9031ef80'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 22:56:53.036860 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-73860959-cdda-4424-9d65-fcda9031ef80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eafb4e17294741e68b70cf9c5e893760', 'request_id': 'req-73860959-cdda-4424-9d65-fcda9031ef80'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 22:56:53.042930 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-73860959-cdda-4424-9d65-fcda9031ef80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eafb4e17294741e68b70cf9c5e893760', 'request_id': 'req-73860959-cdda-4424-9d65-fcda9031ef80'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 22:56:53.059149 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-73860959-cdda-4424-9d65-fcda9031ef80 admin admin] The token cache is being invalidate because project eafb4e17294741e68b70cf9c5e893760 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 22:56:53.059149 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-73860959-cdda-4424-9d65-fcda9031ef80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 22:56:53.059482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-73860959-cdda-4424-9d65-fcda9031ef80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 22:56:53.061028 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 761/1510] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 22:56:52 2022] DELETE /identity/v3/projects/eafb4e17294741e68b70cf9c5e893760 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 22:57:02.605611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:57:03.054935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:58:32.272774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b453d9-5057-4ff5-8fba-5ea7ca4ffa28 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:32.273315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b453d9-5057-4ff5-8fba-5ea7ca4ffa28 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:32.273315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b453d9-5057-4ff5-8fba-5ea7ca4ffa28 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:32.274754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 22:58:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:58:32.279565 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5665cd3-b2c4-40e5-8ced-0c1a67d0e3e7 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:32.279986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5665cd3-b2c4-40e5-8ced-0c1a67d0e3e7 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:32.280167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5665cd3-b2c4-40e5-8ced-0c1a67d0e3e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:32.297906 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-e5665cd3-b2c4-40e5-8ced-0c1a67d0e3e7 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:58:32.299093 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e5665cd3-b2c4-40e5-8ced-0c1a67d0e3e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:58:32.356008 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 22:58:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:58:32.526770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b453d9-5057-4ff5-8fba-5ea7ca4ffa28 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:58:32.528475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-b7b453d9-5057-4ff5-8fba-5ea7ca4ffa28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:58:32.602027 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b453d9-5057-4ff5-8fba-5ea7ca4ffa28 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:58:32.607203 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50bb0d2-d409-413e-a82b-af4a23503495 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:58:32.607847 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bb0d2-d409-413e-a82b-af4a23503495 service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:32.608130 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bb0d2-d409-413e-a82b-af4a23503495 service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:32.608407 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50bb0d2-d409-413e-a82b-af4a23503495 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:32.616869 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50bb0d2-d409-413e-a82b-af4a23503495 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:58:32.618459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50bb0d2-d409-413e-a82b-af4a23503495 service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:58:32.627937 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 751/1513] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:58:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:58:33.520059 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e980b-eeee-4354-bb22-f65b6676b5d7 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:33.520639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e980b-eeee-4354-bb22-f65b6676b5d7 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:33.520639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9e980b-eeee-4354-bb22-f65b6676b5d7 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:33.522107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 22:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:58:33.527132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d6d29-47dd-476f-8aad-83c3795e7b97 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:33.527548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d6d29-47dd-476f-8aad-83c3795e7b97 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:33.527803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98d6d29-47dd-476f-8aad-83c3795e7b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:33.545725 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-b98d6d29-47dd-476f-8aad-83c3795e7b97 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:58:33.546985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-b98d6d29-47dd-476f-8aad-83c3795e7b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:58:33.555511 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 22:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:58:33.710833 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e980b-eeee-4354-bb22-f65b6676b5d7 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:58:33.726194 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9e980b-eeee-4354-bb22-f65b6676b5d7 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:58:33.731603 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee88726-09ef-45fc-8279-d33f88d1adec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:58:33.732253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee88726-09ef-45fc-8279-d33f88d1adec service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:33.732542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee88726-09ef-45fc-8279-d33f88d1adec service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:33.732818 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee88726-09ef-45fc-8279-d33f88d1adec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:33.741558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee88726-09ef-45fc-8279-d33f88d1adec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:58:33.743186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee88726-09ef-45fc-8279-d33f88d1adec service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:58:33.750315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4257d8-fce0-4d89-a4c2-bfed39c2f375 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:33.750629 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4257d8-fce0-4d89-a4c2-bfed39c2f375 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:33.750920 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4257d8-fce0-4d89-a4c2-bfed39c2f375 None None] PATH_INFO: `/v3` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:33.752320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 753/1516] 10.222.0.26 () {54 vars in 925 bytes} [Thu Jun 16 22:58:33 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 22:58:33.753795 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 764/1517] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:58:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:58:33.758307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f4772-3129-4c31-a5b6-b8bc1fe4aa5f None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:33.758616 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f4772-3129-4c31-a5b6-b8bc1fe4aa5f None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:33.758907 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f4772-3129-4c31-a5b6-b8bc1fe4aa5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:33.776579 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-c81f4772-3129-4c31-a5b6-b8bc1fe4aa5f None None] MFA Rules not processed for user `f9593939641c4c7797c1981206b40e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:58:33.777744 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c81f4772-3129-4c31-a5b6-b8bc1fe4aa5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:58:33.846246 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 754/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Thu Jun 16 22:58:33 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:58:37.574935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b078c2b-032a-47a7-bbe2-78d49c8491c8 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:37.575446 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b078c2b-032a-47a7-bbe2-78d49c8491c8 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:37.575446 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b078c2b-032a-47a7-bbe2-78d49c8491c8 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:37.576792 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 765/1519] 10.222.0.17 () {54 vars in 960 bytes} [Thu Jun 16 22:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:58:37.581746 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f3bc8-46c6-4269-9668-1595710e8c7b None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:37.582787 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f3bc8-46c6-4269-9668-1595710e8c7b None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:37.583027 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04f3bc8-46c6-4269-9668-1595710e8c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:37.600811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-e04f3bc8-46c6-4269-9668-1595710e8c7b None None] MFA Rules not processed for user `f9593939641c4c7797c1981206b40e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:58:37.602000 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-e04f3bc8-46c6-4269-9668-1595710e8c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:58:37.610414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 755/1520] 10.222.0.17 () {58 vars in 1057 bytes} [Thu Jun 16 22:58:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:58:37.617471 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b078c2b-032a-47a7-bbe2-78d49c8491c8 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:58:37.631151 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b078c2b-032a-47a7-bbe2-78d49c8491c8 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:58:37.635961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5e4788-bf5a-41d2-a491-cdebb59b5ee4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9593939641c4c7797c1981206b40e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:58:37.636589 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e4788-bf5a-41d2-a491-cdebb59b5ee4 service placement] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:37.636874 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e4788-bf5a-41d2-a491-cdebb59b5ee4 service placement] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:37.637153 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5e4788-bf5a-41d2-a491-cdebb59b5ee4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:37.645754 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e4788-bf5a-41d2-a491-cdebb59b5ee4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:58:37.647345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d5e4788-bf5a-41d2-a491-cdebb59b5ee4 service placement] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:58:37.656656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 766/1521] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:58:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:58:45.025125 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a3957-bef5-4453-9aa7-7d3d45129376 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:45.025695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a3957-bef5-4453-9aa7-7d3d45129376 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:45.025695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a3957-bef5-4453-9aa7-7d3d45129376 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:45.027106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 756/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 22:58:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:58:45.031888 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e602326-e6b3-44fb-881d-f686bc733ce6 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:45.032327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e602326-e6b3-44fb-881d-f686bc733ce6 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:45.032458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e602326-e6b3-44fb-881d-f686bc733ce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:45.050038 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-5e602326-e6b3-44fb-881d-f686bc733ce6 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:58:45.051275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5e602326-e6b3-44fb-881d-f686bc733ce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:58:45.059048 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 767/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 22:58:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:58:45.213102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a3957-bef5-4453-9aa7-7d3d45129376 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:58:45.227071 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a3957-bef5-4453-9aa7-7d3d45129376 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:58:45.232462 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b967a39a-bccc-4686-919c-3d9a21f0a4ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:58:45.233107 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967a39a-bccc-4686-919c-3d9a21f0a4ff service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:45.233391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967a39a-bccc-4686-919c-3d9a21f0a4ff service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:45.233669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b967a39a-bccc-4686-919c-3d9a21f0a4ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:45.241911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b967a39a-bccc-4686-919c-3d9a21f0a4ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:58:45.243521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b967a39a-bccc-4686-919c-3d9a21f0a4ff service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:58:45.253567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 757/1524] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:58:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:58:46.115828 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0475e24-68f3-4e25-ba48-6b288c85ee7a None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:46.116346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0475e24-68f3-4e25-ba48-6b288c85ee7a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:46.116450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0475e24-68f3-4e25-ba48-6b288c85ee7a None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:46.117767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 768/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 22:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:58:46.122707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df26b5-c78c-47c7-b83b-eb99ed7cd08d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:46.123162 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df26b5-c78c-47c7-b83b-eb99ed7cd08d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:46.123268 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66df26b5-c78c-47c7-b83b-eb99ed7cd08d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:46.140870 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-66df26b5-c78c-47c7-b83b-eb99ed7cd08d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:58:46.142062 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-66df26b5-c78c-47c7-b83b-eb99ed7cd08d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:58:46.150615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 758/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 22:58:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:58:46.300647 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0475e24-68f3-4e25-ba48-6b288c85ee7a None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:58:46.314954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0475e24-68f3-4e25-ba48-6b288c85ee7a None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:58:46.320334 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b587d8c-062f-4df0-b380-aa19b17562fb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:58:46.320982 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b587d8c-062f-4df0-b380-aa19b17562fb service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:46.321297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b587d8c-062f-4df0-b380-aa19b17562fb service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:46.321581 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b587d8c-062f-4df0-b380-aa19b17562fb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:46.329657 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b587d8c-062f-4df0-b380-aa19b17562fb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:58:46.331791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b587d8c-062f-4df0-b380-aa19b17562fb service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:58:46.341050 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 769/1527] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:58:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:58:50.227489 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a8430-d4ef-40d5-a469-292abf4ebbc6 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:50.228060 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a8430-d4ef-40d5-a469-292abf4ebbc6 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:50.228060 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854a8430-d4ef-40d5-a469-292abf4ebbc6 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:50.229437 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 759/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 22:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:58:50.235171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985827ed-aed6-491b-8749-ecaa4b459c0b None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:50.235561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985827ed-aed6-491b-8749-ecaa4b459c0b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:50.235735 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985827ed-aed6-491b-8749-ecaa4b459c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:50.254432 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-985827ed-aed6-491b-8749-ecaa4b459c0b None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:58:50.255663 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-985827ed-aed6-491b-8749-ecaa4b459c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:58:50.263415 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 770/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 22:58:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:58:50.300606 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a8430-d4ef-40d5-a469-292abf4ebbc6 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:58:50.302278 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-854a8430-d4ef-40d5-a469-292abf4ebbc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:58:50.353991 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:58:50.358309 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854a8430-d4ef-40d5-a469-292abf4ebbc6 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:58:50.362611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5d386-d197-41fa-88cb-7ff801495eaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:58:50.363284 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5d386-d197-41fa-88cb-7ff801495eaa service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:50.363602 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5d386-d197-41fa-88cb-7ff801495eaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:50.363880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5d386-d197-41fa-88cb-7ff801495eaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:50.373000 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5d386-d197-41fa-88cb-7ff801495eaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:58:50.374415 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5d386-d197-41fa-88cb-7ff801495eaa service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:58:50.383772 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 760/1530] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:58:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:58:52.458145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7cdc2-2ce9-4d9f-ace6-d41fde80496b None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:52.458701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7cdc2-2ce9-4d9f-ace6-d41fde80496b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:52.458856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab7cdc2-2ce9-4d9f-ace6-d41fde80496b None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:52.460126 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 771/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 22:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:58:52.465149 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e88490-0c56-4fc2-90ac-66474c308066 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:52.465594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e88490-0c56-4fc2-90ac-66474c308066 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:52.465735 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e88490-0c56-4fc2-90ac-66474c308066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:52.483895 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-43e88490-0c56-4fc2-90ac-66474c308066 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:58:52.485101 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-43e88490-0c56-4fc2-90ac-66474c308066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:58:52.493551 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 761/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 22:58:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:58:52.528177 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7cdc2-2ce9-4d9f-ace6-d41fde80496b None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:58:52.539612 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab7cdc2-2ce9-4d9f-ace6-d41fde80496b None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:58:52.543123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4526a268-92aa-4c64-bda8-2ce186f2d953 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:58:52.543764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526a268-92aa-4c64-bda8-2ce186f2d953 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:52.544053 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526a268-92aa-4c64-bda8-2ce186f2d953 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:52.544338 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4526a268-92aa-4c64-bda8-2ce186f2d953 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:52.553789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526a268-92aa-4c64-bda8-2ce186f2d953 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:58:52.555234 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4526a268-92aa-4c64-bda8-2ce186f2d953 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:58:52.564515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 772/1533] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:58:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:58:56.126557 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b138223-a7f1-4b3f-aedc-0a2f6f2eb528 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:56.127169 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b138223-a7f1-4b3f-aedc-0a2f6f2eb528 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:56.127169 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b138223-a7f1-4b3f-aedc-0a2f6f2eb528 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:56.128522 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 762/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 22:58:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:58:56.133530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65610c09-4f4b-483b-be64-d6b689584ecd None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:56.133963 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65610c09-4f4b-483b-be64-d6b689584ecd None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:56.134119 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65610c09-4f4b-483b-be64-d6b689584ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:56.140429 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=58 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:58:56.155612 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-65610c09-4f4b-483b-be64-d6b689584ecd None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:58:56.156828 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-65610c09-4f4b-483b-be64-d6b689584ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:58:56.166435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 773/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 22:58:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:58:56.427856 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b138223-a7f1-4b3f-aedc-0a2f6f2eb528 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:58:56.429527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-8b138223-a7f1-4b3f-aedc-0a2f6f2eb528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:58:56.489139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b138223-a7f1-4b3f-aedc-0a2f6f2eb528 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:58:56.492757 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc110e4-de04-40bd-aea3-6213c6ea1ab5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:58:56.493403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc110e4-de04-40bd-aea3-6213c6ea1ab5 service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:56.493687 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc110e4-de04-40bd-aea3-6213c6ea1ab5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:56.494222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc110e4-de04-40bd-aea3-6213c6ea1ab5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:56.503657 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc110e4-de04-40bd-aea3-6213c6ea1ab5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:58:56.505073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc110e4-de04-40bd-aea3-6213c6ea1ab5 service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:58:56.514232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 763/1536] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:58:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:58:58.647718 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d0467-88f3-4c76-88e0-ee4cd733834d None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:58.648292 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d0467-88f3-4c76-88e0-ee4cd733834d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:58.648292 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d0467-88f3-4c76-88e0-ee4cd733834d None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:58.649633 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 774/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 22:58:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 22:58:58.654712 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026bc14-b34e-404d-84f8-c605bb3e111a None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:58.655097 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026bc14-b34e-404d-84f8-c605bb3e111a None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:58.655303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2026bc14-b34e-404d-84f8-c605bb3e111a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:58.674179 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-2026bc14-b34e-404d-84f8-c605bb3e111a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 22:58:58.675428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-2026bc14-b34e-404d-84f8-c605bb3e111a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:58:58.683206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 764/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 22:58:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 22:58:58.953535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d0467-88f3-4c76-88e0-ee4cd733834d None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 22:58:58.955756 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-4b6d0467-88f3-4c76-88e0-ee4cd733834d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 22:58:58.969760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d0467-88f3-4c76-88e0-ee4cd733834d None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 22:58:58.973910 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba92ed1-16e8-4917-b6cd-562722d00bbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 22:58:58.974547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba92ed1-16e8-4917-b6cd-562722d00bbf service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 22:58:58.974865 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba92ed1-16e8-4917-b6cd-562722d00bbf service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 22:58:58.975145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba92ed1-16e8-4917-b6cd-562722d00bbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 22:58:58.984899 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba92ed1-16e8-4917-b6cd-562722d00bbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 22:58:58.986529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba92ed1-16e8-4917-b6cd-562722d00bbf service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 22:58:58.996322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 775/1539] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 22:58:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 22:59:08.668360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=36 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 22:59:08.991886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=8 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:26.695405 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352aec0-bcf4-45f2-a667-2d229f489abb None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:26.696086 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352aec0-bcf4-45f2-a667-2d229f489abb None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:26.696086 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c352aec0-bcf4-45f2-a667-2d229f489abb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:26.728611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-c352aec0-bcf4-45f2-a667-2d229f489abb None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:26.729811 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451f630-2932-4a23-941f-0bea77a8d6a8 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:26.730314 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c352aec0-bcf4-45f2-a667-2d229f489abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:26.730407 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451f630-2932-4a23-941f-0bea77a8d6a8 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:26.730407 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8451f630-2932-4a23-941f-0bea77a8d6a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:26.738616 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 765/1540] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:26.752758 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3c737-391e-4652-89bd-0c459e5da6fb None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:26.753095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3c737-391e-4652-89bd-0c459e5da6fb None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:26.753330 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac3c737-391e-4652-89bd-0c459e5da6fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:26.755489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-8451f630-2932-4a23-941f-0bea77a8d6a8 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:26.756972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-8451f630-2932-4a23-941f-0bea77a8d6a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:26.766240 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 776/1541] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:00:26.777172 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31a71d-1385-4eb5-ac92-829a8b510278 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:26.777771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31a71d-1385-4eb5-ac92-829a8b510278 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:26.778249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31a71d-1385-4eb5-ac92-829a8b510278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:26.787630 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-5ac3c737-391e-4652-89bd-0c459e5da6fb None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:26.789009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-5ac3c737-391e-4652-89bd-0c459e5da6fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:26.802244 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-5d31a71d-1385-4eb5-ac92-829a8b510278 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:26.803807 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5d31a71d-1385-4eb5-ac92-829a8b510278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:26.806020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 766/1542] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:00:26.810390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3c737-391e-4652-89bd-0c459e5da6fb None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:26.818929 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 777/1543] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:00:26.824602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31a71d-1385-4eb5-ac92-829a8b510278 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:26.826585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac3c737-391e-4652-89bd-0c459e5da6fb None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:26.846395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31a71d-1385-4eb5-ac92-829a8b510278 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:26.849963 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a4f57-0d8a-4c9e-aefb-7a65c2136af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:26.850941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a4f57-0d8a-4c9e-aefb-7a65c2136af8 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:26.851307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a4f57-0d8a-4c9e-aefb-7a65c2136af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:26.851739 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3a4f57-0d8a-4c9e-aefb-7a65c2136af8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:26.852620 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a4f57-0d8a-4c9e-aefb-7a65c2136af8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:00:26.853010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a4f57-0d8a-4c9e-aefb-7a65c2136af8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:26.853230 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd215136-f665-4b20-8f39-22a0548c4735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:26.854115 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd215136-f665-4b20-8f39-22a0548c4735 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:26.854250 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: WARNING py.warnings [None req-4d3a4f57-0d8a-4c9e-aefb-7a65c2136af8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 23:00:26.854250 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: warnings.warn(msg) Jun 16 23:00:26.854250 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: Jun 16 23:00:26.854874 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd215136-f665-4b20-8f39-22a0548c4735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:26.855323 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd215136-f665-4b20-8f39-22a0548c4735 admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:26.855678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3a4f57-0d8a-4c9e-aefb-7a65c2136af8 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:26.856412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd215136-f665-4b20-8f39-22a0548c4735 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:00:26.856892 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd215136-f665-4b20-8f39-22a0548c4735 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:26.858716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd215136-f665-4b20-8f39-22a0548c4735 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:26.881235 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 767/1544] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:00:26.883684 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 778/1545] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:00:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 23:00:26.887107 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a4f57-0d8a-4c9e-aefb-7a65c2136af8 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:26.891528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd215136-f665-4b20-8f39-22a0548c4735 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:26.906913 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3a4f57-0d8a-4c9e-aefb-7a65c2136af8 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:26.907157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd215136-f665-4b20-8f39-22a0548c4735 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:26.912366 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa84bf1-a1ad-43e1-a7e8-cf354fcd7c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:26.913394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa84bf1-a1ad-43e1-a7e8-cf354fcd7c18 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:26.913946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa84bf1-a1ad-43e1-a7e8-cf354fcd7c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:26.914450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa84bf1-a1ad-43e1-a7e8-cf354fcd7c18 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:26.915430 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa84bf1-a1ad-43e1-a7e8-cf354fcd7c18 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:26.916989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: WARNING py.warnings [None req-1aa84bf1-a1ad-43e1-a7e8-cf354fcd7c18 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 23:00:26.916989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: warnings.warn(msg) Jun 16 23:00:26.916989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: Jun 16 23:00:26.922619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886eb5e-ba9b-4f84-b872-0841743476b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:26.922619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886eb5e-ba9b-4f84-b872-0841743476b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:26.922619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886eb5e-ba9b-4f84-b872-0841743476b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:26.922619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6886eb5e-ba9b-4f84-b872-0841743476b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:26.923109 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6886eb5e-ba9b-4f84-b872-0841743476b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:26.923109 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6886eb5e-ba9b-4f84-b872-0841743476b1 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:26.924881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa84bf1-a1ad-43e1-a7e8-cf354fcd7c18 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:26.985448 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 779/1546] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:00:26 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:26.989914 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa84bf1-a1ad-43e1-a7e8-cf354fcd7c18 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:26.992958 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 768/1547] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:00:26 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:26.998469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886eb5e-ba9b-4f84-b872-0841743476b1 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:27.079211 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa84bf1-a1ad-43e1-a7e8-cf354fcd7c18 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.152527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6078366-0846-4716-9473-ccf5d4e035c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.153529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6078366-0846-4716-9473-ccf5d4e035c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.153984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6078366-0846-4716-9473-ccf5d4e035c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.154429 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6078366-0846-4716-9473-ccf5d4e035c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.155315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6078366-0846-4716-9473-ccf5d4e035c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.157661 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6078366-0846-4716-9473-ccf5d4e035c6 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:27.160169 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6886eb5e-ba9b-4f84-b872-0841743476b1 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.180663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281307c-3801-4085-a959-c33c028c39fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.181767 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281307c-3801-4085-a959-c33c028c39fc admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.182276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281307c-3801-4085-a959-c33c028c39fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.182817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7281307c-3801-4085-a959-c33c028c39fc admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.183762 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281307c-3801-4085-a959-c33c028c39fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.185007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: WARNING py.warnings [None req-7281307c-3801-4085-a959-c33c028c39fc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 23:00:27.185007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: warnings.warn(msg) Jun 16 23:00:27.185007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: Jun 16 23:00:27.186624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7281307c-3801-4085-a959-c33c028c39fc admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:27.189169 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 780/1548] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:00:26 2022] POST /identity/v3/users => generated 289 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:00:27.194796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcfcf8-b8f6-4548-8ade-e15dfb40e7ef None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.195303 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcfcf8-b8f6-4548-8ade-e15dfb40e7ef None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.196353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdcfcf8-b8f6-4548-8ade-e15dfb40e7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.228600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 769/1549] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:00:26 2022] POST /identity/v3/users => generated 290 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:27.229953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281307c-3801-4085-a959-c33c028c39fc admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:27.239333 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-5bdcfcf8-b8f6-4548-8ade-e15dfb40e7ef None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:27.239333 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5bdcfcf8-b8f6-4548-8ade-e15dfb40e7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:27.252281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7281307c-3801-4085-a959-c33c028c39fc admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.253956 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 781/1550] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:27.257164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62c3317-7649-4174-9816-37911ce989e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.257912 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c3317-7649-4174-9816-37911ce989e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.258876 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c3317-7649-4174-9816-37911ce989e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.259223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62c3317-7649-4174-9816-37911ce989e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.260345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866e94c-883c-467c-b56a-2e7678f23525 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.260487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c3317-7649-4174-9816-37911ce989e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.260635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866e94c-883c-467c-b56a-2e7678f23525 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.260915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866e94c-883c-467c-b56a-2e7678f23525 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.262571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62c3317-7649-4174-9816-37911ce989e8 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:27.274858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 770/1551] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:27.276344 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d4e8a-0a7e-4ed6-a553-9400db7e862d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.276642 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d4e8a-0a7e-4ed6-a553-9400db7e862d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.279823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226d4e8a-0a7e-4ed6-a553-9400db7e862d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.288192 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-3866e94c-883c-467c-b56a-2e7678f23525 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:27.290125 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-3866e94c-883c-467c-b56a-2e7678f23525 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:27.298963 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 782/1552] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:27.299617 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-226d4e8a-0a7e-4ed6-a553-9400db7e862d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:27.299822 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866e94c-883c-467c-b56a-2e7678f23525 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:27.300918 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-226d4e8a-0a7e-4ed6-a553-9400db7e862d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:27.309728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 771/1553] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:27.311005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f914b7-1996-4443-b42e-dbf7f05f9f64 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.311350 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f914b7-1996-4443-b42e-dbf7f05f9f64 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.311633 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f914b7-1996-4443-b42e-dbf7f05f9f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.312296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866e94c-883c-467c-b56a-2e7678f23525 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.317510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d21b639-cda0-4422-9cbc-58c694b7e280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.318219 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21b639-cda0-4422-9cbc-58c694b7e280 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.318517 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21b639-cda0-4422-9cbc-58c694b7e280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.318836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d21b639-cda0-4422-9cbc-58c694b7e280 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.319786 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21b639-cda0-4422-9cbc-58c694b7e280 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.321449 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d21b639-cda0-4422-9cbc-58c694b7e280 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:27.329867 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-78f914b7-1996-4443-b42e-dbf7f05f9f64 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:27.331089 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-78f914b7-1996-4443-b42e-dbf7f05f9f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:27.331800 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 783/1554] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:27.334800 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cfb82-6ed4-4b8f-a5f0-ee5bd36d88ab None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.334800 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cfb82-6ed4-4b8f-a5f0-ee5bd36d88ab None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.334800 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6cfb82-6ed4-4b8f-a5f0-ee5bd36d88ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.346962 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 772/1555] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:27.348027 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f914b7-1996-4443-b42e-dbf7f05f9f64 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:27.351249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-ec6cfb82-6ed4-4b8f-a5f0-ee5bd36d88ab None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:27.352988 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ec6cfb82-6ed4-4b8f-a5f0-ee5bd36d88ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:27.358220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f914b7-1996-4443-b42e-dbf7f05f9f64 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.360863 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 784/1556] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:27.362377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d31195-382b-4f3b-9c37-76c0078961bb None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.362778 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d31195-382b-4f3b-9c37-76c0078961bb None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.363061 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d31195-382b-4f3b-9c37-76c0078961bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.363176 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb694e3-42f9-4c4b-812d-66151b046aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.363835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb694e3-42f9-4c4b-812d-66151b046aad admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.364125 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb694e3-42f9-4c4b-812d-66151b046aad admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.364410 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb694e3-42f9-4c4b-812d-66151b046aad admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.365354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb694e3-42f9-4c4b-812d-66151b046aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.366995 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb694e3-42f9-4c4b-812d-66151b046aad admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:27.377601 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 773/1557] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:27.378869 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b44c6-ea5f-45df-a547-39a88580279a None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.379161 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b44c6-ea5f-45df-a547-39a88580279a None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.379442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e5b44c6-ea5f-45df-a547-39a88580279a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.380982 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-35d31195-382b-4f3b-9c37-76c0078961bb None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:27.382269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-35d31195-382b-4f3b-9c37-76c0078961bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:27.390113 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 785/1558] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:27.391352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe101caf-d92f-41fb-90a2-08556f50a068 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.391647 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe101caf-d92f-41fb-90a2-08556f50a068 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.391935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe101caf-d92f-41fb-90a2-08556f50a068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.398192 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-4e5b44c6-ea5f-45df-a547-39a88580279a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:27.399718 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-4e5b44c6-ea5f-45df-a547-39a88580279a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:27.407773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 774/1559] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:27.408892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d95acb-fa05-4a82-8165-7313fdc99ad5 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.409184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d95acb-fa05-4a82-8165-7313fdc99ad5 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.409465 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d95acb-fa05-4a82-8165-7313fdc99ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.409646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-fe101caf-d92f-41fb-90a2-08556f50a068 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:27.410821 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-fe101caf-d92f-41fb-90a2-08556f50a068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:27.422782 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 786/1560] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:27.423661 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe101caf-d92f-41fb-90a2-08556f50a068 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:27.430658 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:27.431634 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-01d95acb-fa05-4a82-8165-7313fdc99ad5 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:27.433021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-01d95acb-fa05-4a82-8165-7313fdc99ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:27.434635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe101caf-d92f-41fb-90a2-08556f50a068 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.440066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f360b-e67f-44e8-ae46-5367cfaa1d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.440730 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f360b-e67f-44e8-ae46-5367cfaa1d85 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.441027 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f360b-e67f-44e8-ae46-5367cfaa1d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.444968 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 775/1561] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:27.445118 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-133f360b-e67f-44e8-ae46-5367cfaa1d85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.445118 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f360b-e67f-44e8-ae46-5367cfaa1d85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.445314 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6798662-258c-4748-a931-2f309d899a24 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.445314 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6798662-258c-4748-a931-2f309d899a24 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.445314 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6798662-258c-4748-a931-2f309d899a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.445567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-133f360b-e67f-44e8-ae46-5367cfaa1d85 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:27.457798 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 787/1562] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:27.457798 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f360b-e67f-44e8-ae46-5367cfaa1d85 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:27.462098 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-f6798662-258c-4748-a931-2f309d899a24 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:27.463358 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f6798662-258c-4748-a931-2f309d899a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:27.467138 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-133f360b-e67f-44e8-ae46-5367cfaa1d85 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.471431 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 776/1563] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:27.471587 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b1c71-aa36-4730-8d4c-197220f4e21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.472021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6798662-258c-4748-a931-2f309d899a24 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:27.472229 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b1c71-aa36-4730-8d4c-197220f4e21d admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.472512 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b1c71-aa36-4730-8d4c-197220f4e21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.472792 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b1c71-aa36-4730-8d4c-197220f4e21d admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.473500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b1c71-aa36-4730-8d4c-197220f4e21d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:00:27.473806 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b1c71-aa36-4730-8d4c-197220f4e21d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.475421 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b1c71-aa36-4730-8d4c-197220f4e21d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:27.481757 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6798662-258c-4748-a931-2f309d899a24 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.486571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308741ce-6913-4c32-9b80-fcd201dab611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.487472 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308741ce-6913-4c32-9b80-fcd201dab611 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.487763 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308741ce-6913-4c32-9b80-fcd201dab611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.488048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308741ce-6913-4c32-9b80-fcd201dab611 admin admin] PATH_INFO: `/v3/projects/f3b1be8431b0434d8bd6abca293127d5/users/6a993f5d14b64fccad723abecfcadf90/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.494698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 788/1564] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:00:27.494698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aff8a2-f7a0-4661-a3a4-3d1876adca8c None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.494698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aff8a2-f7a0-4661-a3a4-3d1876adca8c None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.494698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aff8a2-f7a0-4661-a3a4-3d1876adca8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.507569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-30aff8a2-f7a0-4661-a3a4-3d1876adca8c None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:27.508787 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-30aff8a2-f7a0-4661-a3a4-3d1876adca8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:27.509187 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308741ce-6913-4c32-9b80-fcd201dab611 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3b1be8431b0434d8bd6abca293127d5, user_id=6a993f5d14b64fccad723abecfcadf90, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.511962 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308741ce-6913-4c32-9b80-fcd201dab611 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:27.516917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 789/1565] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:27.518335 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127d2b9-9c9f-4023-bce7-7bb9a5156c88 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.518664 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127d2b9-9c9f-4023-bce7-7bb9a5156c88 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.519094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127d2b9-9c9f-4023-bce7-7bb9a5156c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.529535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 777/1566] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:00:27 2022] PUT /identity/v3/projects/f3b1be8431b0434d8bd6abca293127d5/users/6a993f5d14b64fccad723abecfcadf90/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:00:27.533190 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308741ce-6913-4c32-9b80-fcd201dab611 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:27.538947 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-4127d2b9-9c9f-4023-bce7-7bb9a5156c88 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:27.540746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-4127d2b9-9c9f-4023-bce7-7bb9a5156c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:27.556117 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:27.593722 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308741ce-6913-4c32-9b80-fcd201dab611 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.598260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd2d4a-ddbc-41c2-8690-cee7acbad880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.598941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd2d4a-ddbc-41c2-8690-cee7acbad880 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.599230 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd2d4a-ddbc-41c2-8690-cee7acbad880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.599341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 790/1567] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:27.599510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cd2d4a-ddbc-41c2-8690-cee7acbad880 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.600179 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127d2b9-9c9f-4023-bce7-7bb9a5156c88 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:27.600317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd2d4a-ddbc-41c2-8690-cee7acbad880 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:00:27.600632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd2d4a-ddbc-41c2-8690-cee7acbad880 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.602783 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cd2d4a-ddbc-41c2-8690-cee7acbad880 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:27.609705 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127d2b9-9c9f-4023-bce7-7bb9a5156c88 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.614577 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146eb02-730f-45b2-b91f-772dffe6d21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.614809 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 778/1568] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:00:27.615253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146eb02-730f-45b2-b91f-772dffe6d21e admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.615545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146eb02-730f-45b2-b91f-772dffe6d21e admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.615657 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd2d4a-ddbc-41c2-8690-cee7acbad880 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:27.615819 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146eb02-730f-45b2-b91f-772dffe6d21e admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.616545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146eb02-730f-45b2-b91f-772dffe6d21e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:00:27.616859 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146eb02-730f-45b2-b91f-772dffe6d21e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.619049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3146eb02-730f-45b2-b91f-772dffe6d21e admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:27.625605 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cd2d4a-ddbc-41c2-8690-cee7acbad880 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.630145 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949421c5-3d33-494b-8db5-9cd9f57078ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.631047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949421c5-3d33-494b-8db5-9cd9f57078ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.631336 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949421c5-3d33-494b-8db5-9cd9f57078ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.631619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949421c5-3d33-494b-8db5-9cd9f57078ee admin admin] PATH_INFO: `/v3/projects/c2300d78ad8244e3be17c1f388603de7/users/a4a6c4623c5f471b8241257180b64d37/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.632712 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 791/1569] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:00:27.633568 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146eb02-730f-45b2-b91f-772dffe6d21e admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:27.644943 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146eb02-730f-45b2-b91f-772dffe6d21e admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.647201 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949421c5-3d33-494b-8db5-9cd9f57078ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2300d78ad8244e3be17c1f388603de7, user_id=a4a6c4623c5f471b8241257180b64d37, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.650142 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b9fca-52ba-4091-8110-656eae4a4a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.650446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949421c5-3d33-494b-8db5-9cd9f57078ee admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:27.650877 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b9fca-52ba-4091-8110-656eae4a4a7d admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.651169 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b9fca-52ba-4091-8110-656eae4a4a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.651452 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4b9fca-52ba-4091-8110-656eae4a4a7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.652172 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b9fca-52ba-4091-8110-656eae4a4a7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:00:27.652484 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b9fca-52ba-4091-8110-656eae4a4a7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.654090 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4b9fca-52ba-4091-8110-656eae4a4a7d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:27.663514 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 779/1570] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:00:27 2022] PUT /identity/v3/projects/c2300d78ad8244e3be17c1f388603de7/users/a4a6c4623c5f471b8241257180b64d37/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:00:27.665152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949421c5-3d33-494b-8db5-9cd9f57078ee admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:27.671492 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 792/1571] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:00:27.672274 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b9fca-52ba-4091-8110-656eae4a4a7d admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:27.681820 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4b9fca-52ba-4091-8110-656eae4a4a7d admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.730790 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949421c5-3d33-494b-8db5-9cd9f57078ee admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.735401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f54f6f-dbb5-462b-b147-dea29019ab76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.736061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f54f6f-dbb5-462b-b147-dea29019ab76 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.736598 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f54f6f-dbb5-462b-b147-dea29019ab76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.736875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f54f6f-dbb5-462b-b147-dea29019ab76 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.737445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9659397e-8a55-427d-957a-0d4db1e72d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.737891 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f54f6f-dbb5-462b-b147-dea29019ab76 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.738114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659397e-8a55-427d-957a-0d4db1e72d4e admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.738403 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659397e-8a55-427d-957a-0d4db1e72d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.738716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9659397e-8a55-427d-957a-0d4db1e72d4e admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.739448 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659397e-8a55-427d-957a-0d4db1e72d4e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:00:27.739558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f54f6f-dbb5-462b-b147-dea29019ab76 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:27.739761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659397e-8a55-427d-957a-0d4db1e72d4e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.741361 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9659397e-8a55-427d-957a-0d4db1e72d4e admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:27.752964 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 793/1572] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:00:27.753924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfd040-ca4f-4f3d-ad8d-e98872d6fd6d None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.754217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfd040-ca4f-4f3d-ad8d-e98872d6fd6d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.754502 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2dfd040-ca4f-4f3d-ad8d-e98872d6fd6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.771176 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 780/1573] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/projects => generated 363 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:27.771397 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a2dfd040-ca4f-4f3d-ad8d-e98872d6fd6d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:27.772427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a2dfd040-ca4f-4f3d-ad8d-e98872d6fd6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:27.772581 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f54f6f-dbb5-462b-b147-dea29019ab76 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:27.810446 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:27.832023 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 794/1574] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:27.835259 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dfd040-ca4f-4f3d-ad8d-e98872d6fd6d None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:27.835365 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f54f6f-dbb5-462b-b147-dea29019ab76 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.838238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79f3bd-9b5b-47f1-bc04-e0a923fab31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.838945 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79f3bd-9b5b-47f1-bc04-e0a923fab31c admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.839235 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79f3bd-9b5b-47f1-bc04-e0a923fab31c admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.839542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad79f3bd-9b5b-47f1-bc04-e0a923fab31c admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.840254 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79f3bd-9b5b-47f1-bc04-e0a923fab31c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:00:27.840574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79f3bd-9b5b-47f1-bc04-e0a923fab31c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.842186 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad79f3bd-9b5b-47f1-bc04-e0a923fab31c admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:27.842834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2dfd040-ca4f-4f3d-ad8d-e98872d6fd6d None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.848266 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1687a5bd-8a22-419d-9f73-a1d24f335ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.848923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687a5bd-8a22-419d-9f73-a1d24f335ce1 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.849674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687a5bd-8a22-419d-9f73-a1d24f335ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.849960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1687a5bd-8a22-419d-9f73-a1d24f335ce1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.850715 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687a5bd-8a22-419d-9f73-a1d24f335ce1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.853325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1687a5bd-8a22-419d-9f73-a1d24f335ce1 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:27.854749 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 781/1575] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:00:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:00:27.855600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79f3bd-9b5b-47f1-bc04-e0a923fab31c admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:27.865961 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad79f3bd-9b5b-47f1-bc04-e0a923fab31c admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.870988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81427fa9-8c87-4c08-8edb-880bdd4ef410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.871732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81427fa9-8c87-4c08-8edb-880bdd4ef410 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.872184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81427fa9-8c87-4c08-8edb-880bdd4ef410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.872527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81427fa9-8c87-4c08-8edb-880bdd4ef410 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.873213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81427fa9-8c87-4c08-8edb-880bdd4ef410 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.875226 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81427fa9-8c87-4c08-8edb-880bdd4ef410 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:27.879353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 795/1576] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:27.880584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25a85a-933d-47e9-8c48-681f1d2d6e78 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.880881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25a85a-933d-47e9-8c48-681f1d2d6e78 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.881165 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25a85a-933d-47e9-8c48-681f1d2d6e78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.887165 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:27.898386 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-2c25a85a-933d-47e9-8c48-681f1d2d6e78 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:27.900222 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-2c25a85a-933d-47e9-8c48-681f1d2d6e78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:27.905976 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 782/1577] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:27.906926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81427fa9-8c87-4c08-8edb-880bdd4ef410 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:27.963290 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 796/1578] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:27.964115 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25a85a-933d-47e9-8c48-681f1d2d6e78 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:27.970763 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81427fa9-8c87-4c08-8edb-880bdd4ef410 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.973094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25a85a-933d-47e9-8c48-681f1d2d6e78 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:27.976029 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88bd10-5cb8-45fe-ba2b-01f666041b55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.976740 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88bd10-5cb8-45fe-ba2b-01f666041b55 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.977013 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88bd10-5cb8-45fe-ba2b-01f666041b55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.977319 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88bd10-5cb8-45fe-ba2b-01f666041b55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.977963 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43919b9f-fb20-401b-8311-6b6a03e53f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:27.978104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88bd10-5cb8-45fe-ba2b-01f666041b55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.978633 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43919b9f-fb20-401b-8311-6b6a03e53f70 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:27.978927 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43919b9f-fb20-401b-8311-6b6a03e53f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:27.979211 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43919b9f-fb20-401b-8311-6b6a03e53f70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:27.979939 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88bd10-5cb8-45fe-ba2b-01f666041b55 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:27.980047 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43919b9f-fb20-401b-8311-6b6a03e53f70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:27.981860 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43919b9f-fb20-401b-8311-6b6a03e53f70 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.007815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 797/1579] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/projects => generated 393 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:28.011315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 783/1580] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:00:27 2022] POST /identity/v3/projects => generated 389 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:28.011942 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43919b9f-fb20-401b-8311-6b6a03e53f70 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.012178 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88bd10-5cb8-45fe-ba2b-01f666041b55 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.013758 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-9c88bd10-5cb8-45fe-ba2b-01f666041b55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:28.068601 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88bd10-5cb8-45fe-ba2b-01f666041b55 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.073097 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43919b9f-fb20-401b-8311-6b6a03e53f70 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.077664 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fef21-a0ee-4410-b432-d4e1007feb69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.078401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fef21-a0ee-4410-b432-d4e1007feb69 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.078702 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fef21-a0ee-4410-b432-d4e1007feb69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.078984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0fef21-a0ee-4410-b432-d4e1007feb69 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.079669 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fef21-a0ee-4410-b432-d4e1007feb69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.081581 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0fef21-a0ee-4410-b432-d4e1007feb69 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.101431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 798/1581] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:00:28 2022] POST /identity/v3/users => generated 282 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:28.105494 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fef21-a0ee-4410-b432-d4e1007feb69 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.115210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0fef21-a0ee-4410-b432-d4e1007feb69 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.119782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab6ffc-bc16-4d12-a3fa-1b6d2f36879b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.120443 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab6ffc-bc16-4d12-a3fa-1b6d2f36879b service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.120575 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57571e10-cf12-460f-a5d7-39cbd873cb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.120850 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab6ffc-bc16-4d12-a3fa-1b6d2f36879b service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.121122 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaab6ffc-bc16-4d12-a3fa-1b6d2f36879b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.121314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57571e10-cf12-460f-a5d7-39cbd873cb3f admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.121602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57571e10-cf12-460f-a5d7-39cbd873cb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.121885 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57571e10-cf12-460f-a5d7-39cbd873cb3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.122567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57571e10-cf12-460f-a5d7-39cbd873cb3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.124491 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57571e10-cf12-460f-a5d7-39cbd873cb3f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.130253 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab6ffc-bc16-4d12-a3fa-1b6d2f36879b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.131781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaab6ffc-bc16-4d12-a3fa-1b6d2f36879b service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.144700 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 784/1582] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:28.145786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab6ffc-bc16-4d12-a3fa-1b6d2f36879b service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.154352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 799/1583] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:00:28 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:28.154352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57571e10-cf12-460f-a5d7-39cbd873cb3f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.156155 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaab6ffc-bc16-4d12-a3fa-1b6d2f36879b service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.164840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:28.195671 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:28.232222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1c39d-e30d-41c3-be1d-269d64302978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.232222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1c39d-e30d-41c3-be1d-269d64302978 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.232222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1c39d-e30d-41c3-be1d-269d64302978 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.232640 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57571e10-cf12-460f-a5d7-39cbd873cb3f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.232743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d1c39d-e30d-41c3-be1d-269d64302978 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.233059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1c39d-e30d-41c3-be1d-269d64302978 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.235436 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d1c39d-e30d-41c3-be1d-269d64302978 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.237929 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8253060-bce6-4233-b85b-eeedb0f32883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.238767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8253060-bce6-4233-b85b-eeedb0f32883 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.239126 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8253060-bce6-4233-b85b-eeedb0f32883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.239403 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8253060-bce6-4233-b85b-eeedb0f32883 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.240281 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8253060-bce6-4233-b85b-eeedb0f32883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.242574 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8253060-bce6-4233-b85b-eeedb0f32883 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.261634 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 785/1584] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:00:28 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:28.262387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1c39d-e30d-41c3-be1d-269d64302978 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.269395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 800/1585] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:00:28 2022] POST /identity/v3/users => generated 290 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:28.270364 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8253060-bce6-4233-b85b-eeedb0f32883 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.280450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8253060-bce6-4233-b85b-eeedb0f32883 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.285756 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47654e-985c-4be4-8358-bd28d620322e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.286440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47654e-985c-4be4-8358-bd28d620322e admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.286763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47654e-985c-4be4-8358-bd28d620322e admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.287046 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d47654e-985c-4be4-8358-bd28d620322e admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.287737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47654e-985c-4be4-8358-bd28d620322e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.290417 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d47654e-985c-4be4-8358-bd28d620322e admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.313085 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 801/1586] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:00:28 2022] POST /identity/v3/users => generated 297 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:28.313905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47654e-985c-4be4-8358-bd28d620322e admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.326903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d47654e-985c-4be4-8358-bd28d620322e admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.334078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c180d-49df-4066-8fdb-102ed5a7f6c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.334078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c180d-49df-4066-8fdb-102ed5a7f6c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.334078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c180d-49df-4066-8fdb-102ed5a7f6c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.334341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d1c39d-e30d-41c3-be1d-269d64302978 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.334421 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151c180d-49df-4066-8fdb-102ed5a7f6c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.334421 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c180d-49df-4066-8fdb-102ed5a7f6c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.335723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151c180d-49df-4066-8fdb-102ed5a7f6c2 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.336911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe55fa8-b614-4ef2-8050-f0f4c28ed5e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.337574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe55fa8-b614-4ef2-8050-f0f4c28ed5e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.337915 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe55fa8-b614-4ef2-8050-f0f4c28ed5e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.338199 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe55fa8-b614-4ef2-8050-f0f4c28ed5e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.351470 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe55fa8-b614-4ef2-8050-f0f4c28ed5e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.353862 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe55fa8-b614-4ef2-8050-f0f4c28ed5e4 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.365864 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 802/1587] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:00:28 2022] POST /identity/v3/users => generated 295 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:28.366878 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 786/1588] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:28.369975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe55fa8-b614-4ef2-8050-f0f4c28ed5e4 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.370112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c180d-49df-4066-8fdb-102ed5a7f6c2 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.379980 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151c180d-49df-4066-8fdb-102ed5a7f6c2 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.380288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe55fa8-b614-4ef2-8050-f0f4c28ed5e4 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.386477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ff1a6-1c4b-49f6-9063-24f963857166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.386662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a3905-7472-4e03-9c7b-552f96140a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.387335 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ff1a6-1c4b-49f6-9063-24f963857166 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.387518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a3905-7472-4e03-9c7b-552f96140a10 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.387657 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ff1a6-1c4b-49f6-9063-24f963857166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.387934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a3905-7472-4e03-9c7b-552f96140a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.388046 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa5ff1a6-1c4b-49f6-9063-24f963857166 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.388383 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7a3905-7472-4e03-9c7b-552f96140a10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.388674 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ff1a6-1c4b-49f6-9063-24f963857166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.389240 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a3905-7472-4e03-9c7b-552f96140a10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.390657 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa5ff1a6-1c4b-49f6-9063-24f963857166 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.391096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7a3905-7472-4e03-9c7b-552f96140a10 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.403992 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 803/1589] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:28.404837 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a3905-7472-4e03-9c7b-552f96140a10 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.418013 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7a3905-7472-4e03-9c7b-552f96140a10 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.418504 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 787/1590] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:00:28 2022] POST /identity/v3/users => generated 287 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:28.419550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ff1a6-1c4b-49f6-9063-24f963857166 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.424649 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a6ba8-ca14-44ad-a602-79ba0d475eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.425616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a6ba8-ca14-44ad-a602-79ba0d475eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.425951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a6ba8-ca14-44ad-a602-79ba0d475eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.426378 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909a6ba8-ca14-44ad-a602-79ba0d475eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.427074 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a6ba8-ca14-44ad-a602-79ba0d475eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.428791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909a6ba8-ca14-44ad-a602-79ba0d475eb8 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.432819 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa5ff1a6-1c4b-49f6-9063-24f963857166 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.437971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12d47e-d007-43c0-9458-1604f3c163b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.438812 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12d47e-d007-43c0-9458-1604f3c163b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.439102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12d47e-d007-43c0-9458-1604f3c163b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.439378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af12d47e-d007-43c0-9458-1604f3c163b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.440093 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12d47e-d007-43c0-9458-1604f3c163b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.441286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 804/1591] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:28.441927 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af12d47e-d007-43c0-9458-1604f3c163b4 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.442388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a6ba8-ca14-44ad-a602-79ba0d475eb8 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.460919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 788/1592] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:28.460919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12d47e-d007-43c0-9458-1604f3c163b4 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.463984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909a6ba8-ca14-44ad-a602-79ba0d475eb8 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.470295 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfbb0b-38c4-4e8b-93c9-2d4fd1026810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.471072 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfbb0b-38c4-4e8b-93c9-2d4fd1026810 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.471598 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfbb0b-38c4-4e8b-93c9-2d4fd1026810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.472117 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56dfbb0b-38c4-4e8b-93c9-2d4fd1026810 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.472940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dfbb0b-38c4-4e8b-93c9-2d4fd1026810 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.475312 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56dfbb0b-38c4-4e8b-93c9-2d4fd1026810 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.475440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af12d47e-d007-43c0-9458-1604f3c163b4 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.482420 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c844a46-91ff-4350-a54f-e2d05272a8c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.483341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c844a46-91ff-4350-a54f-e2d05272a8c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.483614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c844a46-91ff-4350-a54f-e2d05272a8c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.483928 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c844a46-91ff-4350-a54f-e2d05272a8c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.484616 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c844a46-91ff-4350-a54f-e2d05272a8c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.486515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c844a46-91ff-4350-a54f-e2d05272a8c8 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.490095 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 805/1593] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:28.490095 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfbb0b-38c4-4e8b-93c9-2d4fd1026810 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.500042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56dfbb0b-38c4-4e8b-93c9-2d4fd1026810 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.506343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 789/1594] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:28.506343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c844a46-91ff-4350-a54f-e2d05272a8c8 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.507476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee56b0a-8e74-4dfe-abfb-f50d10c9be9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.508788 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee56b0a-8e74-4dfe-abfb-f50d10c9be9f admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.509128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee56b0a-8e74-4dfe-abfb-f50d10c9be9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.509528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee56b0a-8e74-4dfe-abfb-f50d10c9be9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.510390 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee56b0a-8e74-4dfe-abfb-f50d10c9be9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.512196 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee56b0a-8e74-4dfe-abfb-f50d10c9be9f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.518083 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c844a46-91ff-4350-a54f-e2d05272a8c8 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.523515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cde65b-bcc7-4aea-ac7e-3d6b01ed62cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.524354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cde65b-bcc7-4aea-ac7e-3d6b01ed62cc admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.524650 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cde65b-bcc7-4aea-ac7e-3d6b01ed62cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.524952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0cde65b-bcc7-4aea-ac7e-3d6b01ed62cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.525664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cde65b-bcc7-4aea-ac7e-3d6b01ed62cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.526270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 806/1595] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:28.527333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0cde65b-bcc7-4aea-ac7e-3d6b01ed62cc admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.527582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee56b0a-8e74-4dfe-abfb-f50d10c9be9f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.538580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee56b0a-8e74-4dfe-abfb-f50d10c9be9f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.543054 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 790/1596] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:28.543054 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cde65b-bcc7-4aea-ac7e-3d6b01ed62cc admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.546699 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7ae69-7195-4642-915b-1fd4088f6158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.550202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7ae69-7195-4642-915b-1fd4088f6158 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.550202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7ae69-7195-4642-915b-1fd4088f6158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.550202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7ae69-7195-4642-915b-1fd4088f6158 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.550202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7ae69-7195-4642-915b-1fd4088f6158 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.550685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7ae69-7195-4642-915b-1fd4088f6158 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.556844 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0cde65b-bcc7-4aea-ac7e-3d6b01ed62cc admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.563688 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 807/1597] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:28.564521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2c93e-0f7d-4350-8257-5ed455187441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.564677 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7ae69-7195-4642-915b-1fd4088f6158 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.565576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2c93e-0f7d-4350-8257-5ed455187441 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.565999 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2c93e-0f7d-4350-8257-5ed455187441 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.566458 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a2c93e-0f7d-4350-8257-5ed455187441 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.567364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2c93e-0f7d-4350-8257-5ed455187441 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.569162 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a2c93e-0f7d-4350-8257-5ed455187441 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.577485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7ae69-7195-4642-915b-1fd4088f6158 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.590145 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 791/1598] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:28.590145 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2c93e-0f7d-4350-8257-5ed455187441 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.593213 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da83bc-f184-4175-a509-611869f1237c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.593776 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da83bc-f184-4175-a509-611869f1237c admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.594069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da83bc-f184-4175-a509-611869f1237c admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.594349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da83bc-f184-4175-a509-611869f1237c admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.595125 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da83bc-f184-4175-a509-611869f1237c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.596736 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56da83bc-f184-4175-a509-611869f1237c admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.602745 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a2c93e-0f7d-4350-8257-5ed455187441 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.607696 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b54581-2696-4ff2-bcbf-4030a8ffc2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.608518 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b54581-2696-4ff2-bcbf-4030a8ffc2df admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.608880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b54581-2696-4ff2-bcbf-4030a8ffc2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.609238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b54581-2696-4ff2-bcbf-4030a8ffc2df admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.610038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b54581-2696-4ff2-bcbf-4030a8ffc2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.612167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b54581-2696-4ff2-bcbf-4030a8ffc2df admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.619755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 808/1599] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:28.620702 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da83bc-f184-4175-a509-611869f1237c admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.631535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 792/1600] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:28.631983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56da83bc-f184-4175-a509-611869f1237c admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.632483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b54581-2696-4ff2-bcbf-4030a8ffc2df admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.637400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b17df4d-7311-41b4-94ef-edf277edb413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.638573 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17df4d-7311-41b4-94ef-edf277edb413 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.639279 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17df4d-7311-41b4-94ef-edf277edb413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.639279 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b17df4d-7311-41b4-94ef-edf277edb413 admin admin] PATH_INFO: `/v3/projects/a94ff92482e8490e85ee579c2d59d611/users/976e8af4b4b34ad89bbc073ba808eeaa/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.644858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b54581-2696-4ff2-bcbf-4030a8ffc2df admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.650853 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f898e-b798-4d8e-96bf-43168d3df593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.651729 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f898e-b798-4d8e-96bf-43168d3df593 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.652362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f898e-b798-4d8e-96bf-43168d3df593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.652656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767f898e-b798-4d8e-96bf-43168d3df593 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.653677 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f898e-b798-4d8e-96bf-43168d3df593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.655248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b17df4d-7311-41b4-94ef-edf277edb413 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a94ff92482e8490e85ee579c2d59d611, user_id=976e8af4b4b34ad89bbc073ba808eeaa, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.656469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767f898e-b798-4d8e-96bf-43168d3df593 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.656771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: WARNING py.warnings [None req-2b17df4d-7311-41b4-94ef-edf277edb413 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 23:00:28.656771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: warnings.warn(msg) Jun 16 23:00:28.656771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: Jun 16 23:00:28.660020 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b17df4d-7311-41b4-94ef-edf277edb413 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.673007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 793/1601] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:28 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:28.675366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f898e-b798-4d8e-96bf-43168d3df593 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.678492 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 809/1602] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:00:28 2022] PUT /identity/v3/projects/a94ff92482e8490e85ee579c2d59d611/users/976e8af4b4b34ad89bbc073ba808eeaa/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:00:28.680227 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b17df4d-7311-41b4-94ef-edf277edb413 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.703289 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767f898e-b798-4d8e-96bf-43168d3df593 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.739171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:28.802742 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b17df4d-7311-41b4-94ef-edf277edb413 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:28.808192 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e448d-08e0-4f4e-af00-77ea640a50d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.809056 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:28.809756 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e448d-08e0-4f4e-af00-77ea640a50d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.810456 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e448d-08e0-4f4e-af00-77ea640a50d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.811072 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e448d-08e0-4f4e-af00-77ea640a50d0 admin admin] PATH_INFO: `/v3/projects/f3cf903d4bf142e5909f9ea88d639a47/users/1b9b9250d43044e5a20ebd0cea2e9c4d/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.833743 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e448d-08e0-4f4e-af00-77ea640a50d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3cf903d4bf142e5909f9ea88d639a47, user_id=1b9b9250d43044e5a20ebd0cea2e9c4d, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.837558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e448d-08e0-4f4e-af00-77ea640a50d0 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.847475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3861d6-01e5-49e9-a579-53d17e1dcbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:28.849147 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3861d6-01e5-49e9-a579-53d17e1dcbc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:28.849570 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3861d6-01e5-49e9-a579-53d17e1dcbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:28.849985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3861d6-01e5-49e9-a579-53d17e1dcbc3 admin admin] PATH_INFO: `/v3/projects/0526c12e8bc947209e24daf5ddfad7be/users/9e3a2d16133b4af8a208682de8a7d111/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:28.870484 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 810/1603] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:00:28 2022] PUT /identity/v3/projects/f3cf903d4bf142e5909f9ea88d639a47/users/1b9b9250d43044e5a20ebd0cea2e9c4d/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:00:28.882416 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e448d-08e0-4f4e-af00-77ea640a50d0 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.890507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3861d6-01e5-49e9-a579-53d17e1dcbc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0526c12e8bc947209e24daf5ddfad7be, user_id=9e3a2d16133b4af8a208682de8a7d111, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:28.893956 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3861d6-01e5-49e9-a579-53d17e1dcbc3 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:28.916923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 794/1604] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:00:28 2022] PUT /identity/v3/projects/0526c12e8bc947209e24daf5ddfad7be/users/9e3a2d16133b4af8a208682de8a7d111/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:00:28.916923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3861d6-01e5-49e9-a579-53d17e1dcbc3 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:28.962787 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e448d-08e0-4f4e-af00-77ea640a50d0 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:29.008856 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3861d6-01e5-49e9-a579-53d17e1dcbc3 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:29.015467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25dfd3-edea-4945-93ab-9213f81d8ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:29.016557 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25dfd3-edea-4945-93ab-9213f81d8ef7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:29.016967 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25dfd3-edea-4945-93ab-9213f81d8ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:29.017330 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa25dfd3-edea-4945-93ab-9213f81d8ef7 admin admin] PATH_INFO: `/v3/projects/6fc0acf708204e1f8f4510cdb1bbf442/users/a1b1a557a3834b98a3397c41665b5dc6/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:29.037716 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25dfd3-edea-4945-93ab-9213f81d8ef7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fc0acf708204e1f8f4510cdb1bbf442, user_id=a1b1a557a3834b98a3397c41665b5dc6, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:29.040980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa25dfd3-edea-4945-93ab-9213f81d8ef7 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:29.060374 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 795/1605] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:00:28 2022] PUT /identity/v3/projects/6fc0acf708204e1f8f4510cdb1bbf442/users/a1b1a557a3834b98a3397c41665b5dc6/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:00:29.062046 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25dfd3-edea-4945-93ab-9213f81d8ef7 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:29.082044 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c35118f-aded-4f6f-a30b-7d0e55f269b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:29.083341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35118f-aded-4f6f-a30b-7d0e55f269b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:29.083817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35118f-aded-4f6f-a30b-7d0e55f269b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:29.084113 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c35118f-aded-4f6f-a30b-7d0e55f269b4 admin admin] PATH_INFO: `/v3/projects/60447df5461141e883d207e028b74bc7/users/243e592fbf1844d89f55dd5035a684f3/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:29.124326 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35118f-aded-4f6f-a30b-7d0e55f269b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60447df5461141e883d207e028b74bc7, user_id=243e592fbf1844d89f55dd5035a684f3, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:29.127272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c35118f-aded-4f6f-a30b-7d0e55f269b4 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:29.157478 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 811/1606] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:00:28 2022] PUT /identity/v3/projects/60447df5461141e883d207e028b74bc7/users/243e592fbf1844d89f55dd5035a684f3/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:00:29.160168 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10667f7d-7d70-4a55-ac1c-ebc5ccb431cc None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:29.161136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10667f7d-7d70-4a55-ac1c-ebc5ccb431cc None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:29.161343 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10667f7d-7d70-4a55-ac1c-ebc5ccb431cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:29.204347 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-10667f7d-7d70-4a55-ac1c-ebc5ccb431cc None None] MFA Rules not processed for user `976e8af4b4b34ad89bbc073ba808eeaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:29.205740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-10667f7d-7d70-4a55-ac1c-ebc5ccb431cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:29.220265 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa25dfd3-edea-4945-93ab-9213f81d8ef7 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:29.271021 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:29.319805 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:29.330366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ab86c-64a2-4f10-b993-6a33341373c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:29.332512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ab86c-64a2-4f10-b993-6a33341373c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:29.332905 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ab86c-64a2-4f10-b993-6a33341373c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:29.333389 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ab86c-64a2-4f10-b993-6a33341373c3 admin admin] PATH_INFO: `/v3/projects/0701abb69e2f4f3d95a115e4212daa66/users/9e99ede07e7e4622841148ac56398c31/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:29.349973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 812/1607] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:29 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:29.349973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f52e69-181a-4f51-bd40-9ff60085bb8d None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:29.349973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f52e69-181a-4f51-bd40-9ff60085bb8d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:29.349973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f52e69-181a-4f51-bd40-9ff60085bb8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:29.354446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ab86c-64a2-4f10-b993-6a33341373c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0701abb69e2f4f3d95a115e4212daa66, user_id=9e99ede07e7e4622841148ac56398c31, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:29.358176 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ab86c-64a2-4f10-b993-6a33341373c3 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:29.393873 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 796/1608] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:00:29 2022] PUT /identity/v3/projects/0701abb69e2f4f3d95a115e4212daa66/users/9e99ede07e7e4622841148ac56398c31/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:00:29.393873 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc5b59-9e36-4351-8a8e-3dded6c430ee None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:29.393873 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc5b59-9e36-4351-8a8e-3dded6c430ee None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:29.393873 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc5b59-9e36-4351-8a8e-3dded6c430ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:29.416707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-75dc5b59-9e36-4351-8a8e-3dded6c430ee None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:29.418039 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-75dc5b59-9e36-4351-8a8e-3dded6c430ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:29.428363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-f5f52e69-181a-4f51-bd40-9ff60085bb8d None None] MFA Rules not processed for user `1b9b9250d43044e5a20ebd0cea2e9c4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:29.431925 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-f5f52e69-181a-4f51-bd40-9ff60085bb8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:29.500634 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 797/1609] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:29.502391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e925a-d88a-452c-97bb-447d0e913b5a None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:29.503433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e925a-d88a-452c-97bb-447d0e913b5a None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:29.503788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e925a-d88a-452c-97bb-447d0e913b5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:29.540761 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-683e925a-d88a-452c-97bb-447d0e913b5a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:29.542662 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-683e925a-d88a-452c-97bb-447d0e913b5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:29.558166 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 798/1610] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:29.558166 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf5605-49a5-4de9-a3cc-8d6bbf0de2dd None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:29.558166 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf5605-49a5-4de9-a3cc-8d6bbf0de2dd None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:29.558166 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf5605-49a5-4de9-a3cc-8d6bbf0de2dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:29.581127 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-5abf5605-49a5-4de9-a3cc-8d6bbf0de2dd None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:29.581486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 813/1611] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:29 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:29.582314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f52e69-181a-4f51-bd40-9ff60085bb8d None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:29.586496 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-5abf5605-49a5-4de9-a3cc-8d6bbf0de2dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:29.596314 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 799/1612] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:29.599481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542a5c1-3129-4bde-be53-53a0dfb67511 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:29.599481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542a5c1-3129-4bde-be53-53a0dfb67511 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:29.599481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542a5c1-3129-4bde-be53-53a0dfb67511 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:29.599928 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f52e69-181a-4f51-bd40-9ff60085bb8d None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:29.610334 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8c26d-5f83-4e31-8073-ebb27eb470fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:29.611330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8c26d-5f83-4e31-8073-ebb27eb470fd admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:29.611616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8c26d-5f83-4e31-8073-ebb27eb470fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:29.611892 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8c26d-5f83-4e31-8073-ebb27eb470fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:29.612588 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8c26d-5f83-4e31-8073-ebb27eb470fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:29.614491 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8c26d-5f83-4e31-8073-ebb27eb470fd admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:29.634349 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-c542a5c1-3129-4bde-be53-53a0dfb67511 None None] MFA Rules not processed for user `9e99ede07e7e4622841148ac56398c31`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:29.635737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c542a5c1-3129-4bde-be53-53a0dfb67511 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:29.670774 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 814/1613] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:00:29 2022] POST /identity/v3/projects => generated 365 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:29.670774 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8c26d-5f83-4e31-8073-ebb27eb470fd admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:29.729044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 800/1614] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:29 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:29.729902 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542a5c1-3129-4bde-be53-53a0dfb67511 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:29.742521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542a5c1-3129-4bde-be53-53a0dfb67511 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:29.793174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8c26d-5f83-4e31-8073-ebb27eb470fd admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:29.802071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0815ba-3123-45a7-8799-6009353beefa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:29.802779 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0815ba-3123-45a7-8799-6009353beefa service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:29.803072 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0815ba-3123-45a7-8799-6009353beefa service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:29.803370 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0815ba-3123-45a7-8799-6009353beefa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:29.809691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:29.820029 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b6a77-71e6-46d8-8a4e-5da85f03513e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:29.821579 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b6a77-71e6-46d8-8a4e-5da85f03513e admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:29.821882 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b6a77-71e6-46d8-8a4e-5da85f03513e admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:29.822174 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b6a77-71e6-46d8-8a4e-5da85f03513e admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:29.823331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b6a77-71e6-46d8-8a4e-5da85f03513e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:29.827609 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b6a77-71e6-46d8-8a4e-5da85f03513e admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:29.883957 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 801/1615] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:00:29 2022] POST /identity/v3/projects => generated 381 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:29.885198 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b6a77-71e6-46d8-8a4e-5da85f03513e admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:29.919342 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:29.941683 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0815ba-3123-45a7-8799-6009353beefa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:29.946092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0815ba-3123-45a7-8799-6009353beefa service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:29.974097 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 815/1616] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:29.975662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0815ba-3123-45a7-8799-6009353beefa service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:29.987565 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0815ba-3123-45a7-8799-6009353beefa service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:29.990409 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b6a77-71e6-46d8-8a4e-5da85f03513e admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:29.993960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a083510-5877-4026-bc60-dfe78a966663 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:29.995679 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a083510-5877-4026-bc60-dfe78a966663 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:29.996070 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a083510-5877-4026-bc60-dfe78a966663 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:29.996483 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a083510-5877-4026-bc60-dfe78a966663 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:29.999297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a5da5-2302-43fc-8438-42eac9555f81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:30.000072 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a5da5-2302-43fc-8438-42eac9555f81 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:30.000355 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a5da5-2302-43fc-8438-42eac9555f81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:30.000649 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a5da5-2302-43fc-8438-42eac9555f81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:30.078746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a083510-5877-4026-bc60-dfe78a966663 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:30.084677 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a083510-5877-4026-bc60-dfe78a966663 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:30.099174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 816/1617] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:30.100017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a083510-5877-4026-bc60-dfe78a966663 service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:30.111829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a083510-5877-4026-bc60-dfe78a966663 service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:30.118157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424a05d-f0ff-444e-9d38-e1744f414969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:30.118877 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424a05d-f0ff-444e-9d38-e1744f414969 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:30.119166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424a05d-f0ff-444e-9d38-e1744f414969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:30.119442 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0424a05d-f0ff-444e-9d38-e1744f414969 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:30.120237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424a05d-f0ff-444e-9d38-e1744f414969 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:30.123382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a5da5-2302-43fc-8438-42eac9555f81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:30.123546 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0424a05d-f0ff-444e-9d38-e1744f414969 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:30.125592 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a5da5-2302-43fc-8438-42eac9555f81 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:30.142247 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 802/1618] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:30.144238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a5da5-2302-43fc-8438-42eac9555f81 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:30.167456 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 817/1619] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:00:30 2022] POST /identity/v3/users => generated 289 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:30.168777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424a05d-f0ff-444e-9d38-e1744f414969 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:30.173793 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a5da5-2302-43fc-8438-42eac9555f81 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:30.182103 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0424a05d-f0ff-444e-9d38-e1744f414969 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:30.188300 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45633301-6a48-470f-924f-865417be8b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:30.188531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c649c-1700-4a2e-82b8-24b81d5090c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:30.189053 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45633301-6a48-470f-924f-865417be8b62 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:30.189207 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c649c-1700-4a2e-82b8-24b81d5090c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:30.189341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45633301-6a48-470f-924f-865417be8b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:30.189503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c649c-1700-4a2e-82b8-24b81d5090c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:30.190755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99c649c-1700-4a2e-82b8-24b81d5090c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:30.191471 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c649c-1700-4a2e-82b8-24b81d5090c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:30.192103 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45633301-6a48-470f-924f-865417be8b62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:30.193981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45633301-6a48-470f-924f-865417be8b62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:30.194985 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99c649c-1700-4a2e-82b8-24b81d5090c7 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:30.198515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45633301-6a48-470f-924f-865417be8b62 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:30.234032 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 818/1620] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:00:30 2022] POST /identity/v3/users => generated 297 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:30.241895 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c649c-1700-4a2e-82b8-24b81d5090c7 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:30.244175 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 803/1621] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:00:30 2022] POST /identity/v3/projects => generated 371 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:30.245328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45633301-6a48-470f-924f-865417be8b62 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:30.263179 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99c649c-1700-4a2e-82b8-24b81d5090c7 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:30.333594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45633301-6a48-470f-924f-865417be8b62 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:30.342056 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a13e448-440e-438e-ab3e-91714c088970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:30.342960 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13e448-440e-438e-ab3e-91714c088970 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:30.343164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13e448-440e-438e-ab3e-91714c088970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:30.343407 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a13e448-440e-438e-ab3e-91714c088970 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:30.344353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a13e448-440e-438e-ab3e-91714c088970 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:30.347247 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a13e448-440e-438e-ab3e-91714c088970 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:30.366507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 804/1622] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:30.367844 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a13e448-440e-438e-ab3e-91714c088970 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:30.378382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:30.384731 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a13e448-440e-438e-ab3e-91714c088970 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:30.392237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d88f8-6460-47f5-9c3a-b51ed9d24fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:30.393232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e3c77-c07c-45cf-92b4-77b4091829f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:30.393504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d88f8-6460-47f5-9c3a-b51ed9d24fca admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:30.394011 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e3c77-c07c-45cf-92b4-77b4091829f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:30.394380 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d88f8-6460-47f5-9c3a-b51ed9d24fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:30.394652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e3c77-c07c-45cf-92b4-77b4091829f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:30.394948 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e3c77-c07c-45cf-92b4-77b4091829f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:30.395797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e3c77-c07c-45cf-92b4-77b4091829f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:30.400567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e3c77-c07c-45cf-92b4-77b4091829f8 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:30.400815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462d88f8-6460-47f5-9c3a-b51ed9d24fca admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:30.402710 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d88f8-6460-47f5-9c3a-b51ed9d24fca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:30.406830 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-462d88f8-6460-47f5-9c3a-b51ed9d24fca admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:30.424233 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 819/1623] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:30.425443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d88f8-6460-47f5-9c3a-b51ed9d24fca admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:30.437869 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 805/1624] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:00:30 2022] POST /identity/v3/users => generated 292 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:00:30.439762 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e3c77-c07c-45cf-92b4-77b4091829f8 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:30.440021 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462d88f8-6460-47f5-9c3a-b51ed9d24fca admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:30.447061 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18702e-4041-4807-aefc-b8e06dce1673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:30.449145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18702e-4041-4807-aefc-b8e06dce1673 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:30.449872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18702e-4041-4807-aefc-b8e06dce1673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:30.450435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e18702e-4041-4807-aefc-b8e06dce1673 admin admin] PATH_INFO: `/v3/projects/db045edf3677494ab6117428fd013c73/users/8b6b04b804df45f897aade6dfc76595e/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:30.462461 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e3c77-c07c-45cf-92b4-77b4091829f8 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:30.470839 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a835e-02d4-4f0b-980c-a95b17252490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:30.472701 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a835e-02d4-4f0b-980c-a95b17252490 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:30.472701 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a835e-02d4-4f0b-980c-a95b17252490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:30.472701 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a835e-02d4-4f0b-980c-a95b17252490 admin admin] PATH_INFO: `/v3/projects/6d06ad8f0ca44a6db60ba1ce6b0450b9/users/a344653f0f0946038ef37ae2c16ab779/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:30.480853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18702e-4041-4807-aefc-b8e06dce1673 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db045edf3677494ab6117428fd013c73, user_id=8b6b04b804df45f897aade6dfc76595e, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:30.485314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e18702e-4041-4807-aefc-b8e06dce1673 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:30.505900 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a835e-02d4-4f0b-980c-a95b17252490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d06ad8f0ca44a6db60ba1ce6b0450b9, user_id=a344653f0f0946038ef37ae2c16ab779, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:30.511486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a835e-02d4-4f0b-980c-a95b17252490 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:30.515763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 820/1625] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:00:30 2022] PUT /identity/v3/projects/db045edf3677494ab6117428fd013c73/users/8b6b04b804df45f897aade6dfc76595e/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:00:30.517436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18702e-4041-4807-aefc-b8e06dce1673 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:30.543976 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 806/1626] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:00:30 2022] PUT /identity/v3/projects/6d06ad8f0ca44a6db60ba1ce6b0450b9/users/a344653f0f0946038ef37ae2c16ab779/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:00:30.544900 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a835e-02d4-4f0b-980c-a95b17252490 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:30.564399 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:30.656300 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e18702e-4041-4807-aefc-b8e06dce1673 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:30.691480 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a835e-02d4-4f0b-980c-a95b17252490 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:30.707590 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ece59-6e0d-4eb2-9eb8-7d709671d6c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:30.709065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ece59-6e0d-4eb2-9eb8-7d709671d6c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:30.709637 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ece59-6e0d-4eb2-9eb8-7d709671d6c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:30.712007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4ece59-6e0d-4eb2-9eb8-7d709671d6c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:30.713047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ece59-6e0d-4eb2-9eb8-7d709671d6c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:30.717184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4ece59-6e0d-4eb2-9eb8-7d709671d6c3 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:30.749640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 807/1627] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:30.749640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ece59-6e0d-4eb2-9eb8-7d709671d6c3 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:30.772744 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4ece59-6e0d-4eb2-9eb8-7d709671d6c3 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:30.783665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9488570-14ba-4b55-a611-a74ef9350d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:30.784110 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07474567-4cae-4719-9d8d-12219ca6dc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:30.784256 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9488570-14ba-4b55-a611-a74ef9350d97 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:30.784577 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9488570-14ba-4b55-a611-a74ef9350d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:30.784859 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9488570-14ba-4b55-a611-a74ef9350d97 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:30.785653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9488570-14ba-4b55-a611-a74ef9350d97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:30.785892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07474567-4cae-4719-9d8d-12219ca6dc91 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:30.786329 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07474567-4cae-4719-9d8d-12219ca6dc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:30.786766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07474567-4cae-4719-9d8d-12219ca6dc91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:30.787955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07474567-4cae-4719-9d8d-12219ca6dc91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:30.790138 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07474567-4cae-4719-9d8d-12219ca6dc91 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:30.791254 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9488570-14ba-4b55-a611-a74ef9350d97 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:30.806143 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 808/1628] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:30.809166 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07474567-4cae-4719-9d8d-12219ca6dc91 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:30.816603 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 821/1629] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:30 2022] GET /identity/v3/roles => generated 1420 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:30.817453 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9488570-14ba-4b55-a611-a74ef9350d97 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:30.825212 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07474567-4cae-4719-9d8d-12219ca6dc91 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:30.834072 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a84d27-8bf8-4e34-be4f-849c0006c500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:30.842037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a84d27-8bf8-4e34-be4f-849c0006c500 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:30.843743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a84d27-8bf8-4e34-be4f-849c0006c500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:30.844218 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a84d27-8bf8-4e34-be4f-849c0006c500 admin admin] PATH_INFO: `/v3/projects/db045edf3677494ab6117428fd013c73/users/8b6b04b804df45f897aade6dfc76595e/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:30.847018 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9488570-14ba-4b55-a611-a74ef9350d97 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:30.847234 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a84d27-8bf8-4e34-be4f-849c0006c500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db045edf3677494ab6117428fd013c73, user_id=8b6b04b804df45f897aade6dfc76595e, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:30.857180 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a84d27-8bf8-4e34-be4f-849c0006c500 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:30.862443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c85d47-5348-4d01-99a8-d0d569927eea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:30.863459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c85d47-5348-4d01-99a8-d0d569927eea admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:30.863755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c85d47-5348-4d01-99a8-d0d569927eea admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:30.864081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c85d47-5348-4d01-99a8-d0d569927eea admin admin] PATH_INFO: `/v3/projects/6d06ad8f0ca44a6db60ba1ce6b0450b9/users/a344653f0f0946038ef37ae2c16ab779/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:30.872542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c85d47-5348-4d01-99a8-d0d569927eea admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d06ad8f0ca44a6db60ba1ce6b0450b9, user_id=a344653f0f0946038ef37ae2c16ab779, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:30.875586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c85d47-5348-4d01-99a8-d0d569927eea admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:30.887515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 809/1630] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:00:30 2022] PUT /identity/v3/projects/db045edf3677494ab6117428fd013c73/users/8b6b04b804df45f897aade6dfc76595e/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:00:30.909067 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 822/1631] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:00:30 2022] PUT /identity/v3/projects/6d06ad8f0ca44a6db60ba1ce6b0450b9/users/a344653f0f0946038ef37ae2c16ab779/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:00:30.910760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd22241-ed75-45e2-80b9-62742c2c9dd4 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:30.911107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd22241-ed75-45e2-80b9-62742c2c9dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:30.911399 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd22241-ed75-45e2-80b9-62742c2c9dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:30.912976 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a84d27-8bf8-4e34-be4f-849c0006c500 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:30.958229 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-2cd22241-ed75-45e2-80b9-62742c2c9dd4 None None] MFA Rules not processed for user `8b6b04b804df45f897aade6dfc76595e`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:30.960191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-2cd22241-ed75-45e2-80b9-62742c2c9dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:31.113747 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a84d27-8bf8-4e34-be4f-849c0006c500 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:31.119999 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 823/1632] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:31.119999 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d9483-4e47-40d8-a82e-d307b0f27b78 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:31.119999 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d9483-4e47-40d8-a82e-d307b0f27b78 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:31.119999 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425d9483-4e47-40d8-a82e-d307b0f27b78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:31.183956 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-425d9483-4e47-40d8-a82e-d307b0f27b78 None None] MFA Rules not processed for user `a344653f0f0946038ef37ae2c16ab779`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:31.185547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-425d9483-4e47-40d8-a82e-d307b0f27b78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:31.217224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:31.235163 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:31.334880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b048c-7bd6-47df-9d35-4dc027872389 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:31.334880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b048c-7bd6-47df-9d35-4dc027872389 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:31.334880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b048c-7bd6-47df-9d35-4dc027872389 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:31.334880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b048c-7bd6-47df-9d35-4dc027872389 admin admin] PATH_INFO: `/v3/projects/ed0a26b2ea754016b1fc625c1e6d8bc8/users/95733192f45d43b89b1f2e3ac9a5f14e/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:31.335498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b048c-7bd6-47df-9d35-4dc027872389 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed0a26b2ea754016b1fc625c1e6d8bc8, user_id=95733192f45d43b89b1f2e3ac9a5f14e, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:31.335498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b048c-7bd6-47df-9d35-4dc027872389 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:31.339706 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 810/1633] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:00:30 2022] PUT /identity/v3/projects/ed0a26b2ea754016b1fc625c1e6d8bc8/users/95733192f45d43b89b1f2e3ac9a5f14e/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:00:31.343720 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b048c-7bd6-47df-9d35-4dc027872389 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:31.372497 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 824/1634] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:31 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:31.373800 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d9483-4e47-40d8-a82e-d307b0f27b78 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:31.403099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425d9483-4e47-40d8-a82e-d307b0f27b78 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:31.461956 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b048c-7bd6-47df-9d35-4dc027872389 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:31.473390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1a65e-2cd9-459d-9d29-e5643c4720da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:31.474707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1a65e-2cd9-459d-9d29-e5643c4720da service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:31.475607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1a65e-2cd9-459d-9d29-e5643c4720da service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:31.477025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1a65e-2cd9-459d-9d29-e5643c4720da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:31.603421 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c10ee-de30-4629-951e-2d052fe47efd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:31.604422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c10ee-de30-4629-951e-2d052fe47efd admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:31.604868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c10ee-de30-4629-951e-2d052fe47efd admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:31.605313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385c10ee-de30-4629-951e-2d052fe47efd admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:31.606219 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c10ee-de30-4629-951e-2d052fe47efd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:31.608070 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385c10ee-de30-4629-951e-2d052fe47efd admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:31.610032 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1a65e-2cd9-459d-9d29-e5643c4720da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:31.612462 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1a65e-2cd9-459d-9d29-e5643c4720da service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:31.621868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 825/1635] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:00:31 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:00:31.623349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c10ee-de30-4629-951e-2d052fe47efd admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:31.629701 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 811/1636] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:31.630778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1a65e-2cd9-459d-9d29-e5643c4720da service cinder] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:31.648631 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1a65e-2cd9-459d-9d29-e5643c4720da service cinder] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:31.652544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385c10ee-de30-4629-951e-2d052fe47efd admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:31.660265 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5efc38-d633-4e3e-acf4-200b0c6841b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:31.661302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5efc38-d633-4e3e-acf4-200b0c6841b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:31.661789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5efc38-d633-4e3e-acf4-200b0c6841b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:31.662380 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5efc38-d633-4e3e-acf4-200b0c6841b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:31.664215 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443aeb3-221b-481a-9d68-fa8b68e2d3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:31.665364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443aeb3-221b-481a-9d68-fa8b68e2d3f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:31.665660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443aeb3-221b-481a-9d68-fa8b68e2d3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:31.665946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443aeb3-221b-481a-9d68-fa8b68e2d3f1 admin admin] PATH_INFO: `/v3/projects/ed0a26b2ea754016b1fc625c1e6d8bc8/users/95733192f45d43b89b1f2e3ac9a5f14e/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:31.668172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443aeb3-221b-481a-9d68-fa8b68e2d3f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed0a26b2ea754016b1fc625c1e6d8bc8, user_id=95733192f45d43b89b1f2e3ac9a5f14e, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:31.671118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2443aeb3-221b-481a-9d68-fa8b68e2d3f1 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:31.687603 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 812/1637] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:00:31 2022] PUT /identity/v3/projects/ed0a26b2ea754016b1fc625c1e6d8bc8/users/95733192f45d43b89b1f2e3ac9a5f14e/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:00:31.706268 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d34ad-6074-491d-9bb3-8e339efa5af9 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:31.706847 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d34ad-6074-491d-9bb3-8e339efa5af9 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:31.707135 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869d34ad-6074-491d-9bb3-8e339efa5af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:31.768911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-869d34ad-6074-491d-9bb3-8e339efa5af9 None None] MFA Rules not processed for user `95733192f45d43b89b1f2e3ac9a5f14e`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:31.770331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-869d34ad-6074-491d-9bb3-8e339efa5af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:31.823067 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5efc38-d633-4e3e-acf4-200b0c6841b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:31.826066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5efc38-d633-4e3e-acf4-200b0c6841b4 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:31.839776 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 826/1638] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:31 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:31.840822 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5efc38-d633-4e3e-acf4-200b0c6841b4 service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:31.852439 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:31.855435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5efc38-d633-4e3e-acf4-200b0c6841b4 service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:31.887793 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 813/1639] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:31 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:31.914478 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d34ad-6074-491d-9bb3-8e339efa5af9 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:31.934859 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f36635-9d3d-4a30-8cbd-5491f409f64f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:31.935686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f36635-9d3d-4a30-8cbd-5491f409f64f service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:31.936200 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f36635-9d3d-4a30-8cbd-5491f409f64f service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:31.936585 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f36635-9d3d-4a30-8cbd-5491f409f64f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:31.942921 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:32.015358 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f36635-9d3d-4a30-8cbd-5491f409f64f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:32.017001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f36635-9d3d-4a30-8cbd-5491f409f64f service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:32.018310 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869d34ad-6074-491d-9bb3-8e339efa5af9 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:32.024575 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe23967-84a5-48f2-bf5e-55caea30a0b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:32.024907 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe23967-84a5-48f2-bf5e-55caea30a0b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:32.025273 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe23967-84a5-48f2-bf5e-55caea30a0b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:32.025762 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe23967-84a5-48f2-bf5e-55caea30a0b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:32.029251 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 827/1640] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:31 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:32.043848 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe23967-84a5-48f2-bf5e-55caea30a0b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:32.045379 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe23967-84a5-48f2-bf5e-55caea30a0b4 service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:32.070521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 814/1641] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:31 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:00:32.259508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f36635-9d3d-4a30-8cbd-5491f409f64f service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:32.272296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f36635-9d3d-4a30-8cbd-5491f409f64f service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:32.279608 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdba324-1d1c-410e-8dbb-176fdba76803 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:32.280504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdba324-1d1c-410e-8dbb-176fdba76803 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:32.281066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdba324-1d1c-410e-8dbb-176fdba76803 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:32.281419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdba324-1d1c-410e-8dbb-176fdba76803 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:32.389993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdba324-1d1c-410e-8dbb-176fdba76803 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:32.391527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdba324-1d1c-410e-8dbb-176fdba76803 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:32.419928 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 828/1642] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:32 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:32.470856 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe23967-84a5-48f2-bf5e-55caea30a0b4 service cinder] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:32.482348 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe23967-84a5-48f2-bf5e-55caea30a0b4 service cinder] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:32.499758 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4339b7-44ce-4afe-8899-1e8f06681c2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:32.505727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4339b7-44ce-4afe-8899-1e8f06681c2b service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:32.508468 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4339b7-44ce-4afe-8899-1e8f06681c2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:32.508804 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4339b7-44ce-4afe-8899-1e8f06681c2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:32.758620 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4339b7-44ce-4afe-8899-1e8f06681c2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:32.762503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4339b7-44ce-4afe-8899-1e8f06681c2b service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:32.796969 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 815/1643] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:39.307739 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed280a8-a33d-4e98-85dc-cd8bbc328ff3 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:39.308279 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed280a8-a33d-4e98-85dc-cd8bbc328ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:39.308279 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed280a8-a33d-4e98-85dc-cd8bbc328ff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:39.357660 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-bed280a8-a33d-4e98-85dc-cd8bbc328ff3 None None] MFA Rules not processed for user `a4a6c4623c5f471b8241257180b64d37`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:39.360601 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-bed280a8-a33d-4e98-85dc-cd8bbc328ff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:39.504642 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b150a1-1472-45ab-abb8-40173adec5a9 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:39.505151 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b150a1-1472-45ab-abb8-40173adec5a9 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:39.505418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b150a1-1472-45ab-abb8-40173adec5a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:39.524582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 829/1644] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:39.543720 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-f2b150a1-1472-45ab-abb8-40173adec5a9 None None] MFA Rules not processed for user `a4a6c4623c5f471b8241257180b64d37`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:39.545242 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f2b150a1-1472-45ab-abb8-40173adec5a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:39.581774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 816/1645] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:39 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:39.634154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e99ff-c9a2-4171-86d2-510abc0ce295 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:39.634655 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e99ff-c9a2-4171-86d2-510abc0ce295 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:39.635627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7e99ff-c9a2-4171-86d2-510abc0ce295 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:39.638964 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 830/1646] 10.222.0.17 () {54 vars in 961 bytes} [Thu Jun 16 23:00:39 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 23:00:39.668811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e8e37-facf-4fd0-b0e5-d97f0bf1beaa None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:39.669495 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e8e37-facf-4fd0-b0e5-d97f0bf1beaa None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:39.670052 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e8e37-facf-4fd0-b0e5-d97f0bf1beaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:39.724000 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-5a9e8e37-facf-4fd0-b0e5-d97f0bf1beaa None None] MFA Rules not processed for user `4180f2593149437fa27effd8d1338d10`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:39.725295 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-5a9e8e37-facf-4fd0-b0e5-d97f0bf1beaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:39.836417 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e99ff-c9a2-4171-86d2-510abc0ce295 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:39.839309 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 817/1647] 10.222.0.17 () {58 vars in 1058 bytes} [Thu Jun 16 23:00:39 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:00:39.855796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7e99ff-c9a2-4171-86d2-510abc0ce295 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:39.863643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029b3181-9f92-4093-95bb-a4a586e07b4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:39.864329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b3181-9f92-4093-95bb-a4a586e07b4b service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:39.864613 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b3181-9f92-4093-95bb-a4a586e07b4b service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:39.864889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029b3181-9f92-4093-95bb-a4a586e07b4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:39.878872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b3181-9f92-4093-95bb-a4a586e07b4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:39.880433 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029b3181-9f92-4093-95bb-a4a586e07b4b service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:39.912579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 831/1648] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:00:40.506666 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54654db6-2bec-45d4-aa52-558eb2bea609 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:40.507125 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54654db6-2bec-45d4-aa52-558eb2bea609 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:40.507237 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54654db6-2bec-45d4-aa52-558eb2bea609 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:40.508647 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 818/1649] 10.222.0.17 () {54 vars in 912 bytes} [Thu Jun 16 23:00:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 23:00:40.515564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f82221-ce03-40df-86a3-013fba18b8a7 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:40.516017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f82221-ce03-40df-86a3-013fba18b8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:40.516017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f82221-ce03-40df-86a3-013fba18b8a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:40.559254 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-e1f82221-ce03-40df-86a3-013fba18b8a7 None None] MFA Rules not processed for user `4180f2593149437fa27effd8d1338d10`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:40.560516 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e1f82221-ce03-40df-86a3-013fba18b8a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:40.593708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 832/1650] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 23:00:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:40.611867 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54654db6-2bec-45d4-aa52-558eb2bea609 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:40.615646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-54654db6-2bec-45d4-aa52-558eb2bea609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:40.644497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:40.792006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54654db6-2bec-45d4-aa52-558eb2bea609 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:40.801309 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f880a9-2d47-4b3b-887d-90cc275c6c44 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915f4296504b47d796e3a5b99550eb75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:40.801993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f880a9-2d47-4b3b-887d-90cc275c6c44 service glance] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:40.804247 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f880a9-2d47-4b3b-887d-90cc275c6c44 service glance] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:40.804546 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f880a9-2d47-4b3b-887d-90cc275c6c44 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:40.849221 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f880a9-2d47-4b3b-887d-90cc275c6c44 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:40.851537 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f880a9-2d47-4b3b-887d-90cc275c6c44 service glance] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:40.891919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 819/1651] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:41.719720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8890a10-1f5c-4806-80aa-ebf629754f8e None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:41.720174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8890a10-1f5c-4806-80aa-ebf629754f8e None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:41.720257 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8890a10-1f5c-4806-80aa-ebf629754f8e None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:41.722082 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 833/1652] 10.222.0.17 () {54 vars in 912 bytes} [Thu Jun 16 23:00:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 23:00:41.728176 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acd92b-4243-4dc4-b4c3-4c9b369378dd None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:41.728662 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acd92b-4243-4dc4-b4c3-4c9b369378dd None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:41.729025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26acd92b-4243-4dc4-b4c3-4c9b369378dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:41.816731 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-26acd92b-4243-4dc4-b4c3-4c9b369378dd None None] MFA Rules not processed for user `6b794f7aa2a7488e91f7d9ea35a45d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:41.821890 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-26acd92b-4243-4dc4-b4c3-4c9b369378dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:41.948407 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 820/1653] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 23:00:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:42.207086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8890a10-1f5c-4806-80aa-ebf629754f8e None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:42.225337 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8890a10-1f5c-4806-80aa-ebf629754f8e None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:42.234402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692efcc-b2de-46e1-a924-afc7caf571e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:42.235461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692efcc-b2de-46e1-a924-afc7caf571e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:42.235799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692efcc-b2de-46e1-a924-afc7caf571e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:42.236149 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692efcc-b2de-46e1-a924-afc7caf571e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:42.269373 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692efcc-b2de-46e1-a924-afc7caf571e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:42.272402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692efcc-b2de-46e1-a924-afc7caf571e2 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:42.300387 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 834/1654] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:42.964627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5de722-80ce-45bc-a356-5a31fef0e1ee None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:42.965122 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5de722-80ce-45bc-a356-5a31fef0e1ee None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:42.965318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5de722-80ce-45bc-a356-5a31fef0e1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:43.018094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149e8cd-59a5-4b16-9530-a98e5e135b44 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:43.018644 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149e8cd-59a5-4b16-9530-a98e5e135b44 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:43.019522 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149e8cd-59a5-4b16-9530-a98e5e135b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:43.027513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:43.039044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-4d5de722-80ce-45bc-a356-5a31fef0e1ee None None] MFA Rules not processed for user `6a993f5d14b64fccad723abecfcadf90`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:43.040422 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-4d5de722-80ce-45bc-a356-5a31fef0e1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:43.112519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-6149e8cd-59a5-4b16-9530-a98e5e135b44 None None] MFA Rules not processed for user `243e592fbf1844d89f55dd5035a684f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:43.113773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-6149e8cd-59a5-4b16-9530-a98e5e135b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:43.250376 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 821/1655] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:43.276293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5de722-80ce-45bc-a356-5a31fef0e1ee None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:43.297737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5de722-80ce-45bc-a356-5a31fef0e1ee None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:43.307135 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c7c205-98f5-4f0a-be6d-67efcbfd5506 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:43.307963 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7c205-98f5-4f0a-be6d-67efcbfd5506 service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:43.308257 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7c205-98f5-4f0a-be6d-67efcbfd5506 service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:43.321803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c7c205-98f5-4f0a-be6d-67efcbfd5506 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:43.358835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7c205-98f5-4f0a-be6d-67efcbfd5506 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:43.376488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c7c205-98f5-4f0a-be6d-67efcbfd5506 service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:43.410531 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 822/1656] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:43.453089 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f2f36-0f78-4fdd-a360-079d3cf4ef29 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:43.453389 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f2f36-0f78-4fdd-a360-079d3cf4ef29 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:43.455426 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5f2f36-0f78-4fdd-a360-079d3cf4ef29 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:43.455867 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 823/1657] 10.222.0.17 () {54 vars in 960 bytes} [Thu Jun 16 23:00:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 23:00:43.461233 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e00e3-738a-4ae4-a810-5f60086f0703 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:43.461527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e00e3-738a-4ae4-a810-5f60086f0703 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:43.461809 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22e00e3-738a-4ae4-a810-5f60086f0703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:43.516548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-e22e00e3-738a-4ae4-a810-5f60086f0703 None None] MFA Rules not processed for user `f9593939641c4c7797c1981206b40e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:43.517743 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 835/1658] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 501 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:43.530950 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c160781-9a81-4506-869b-411cff0e61b2 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:43.531151 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-e22e00e3-738a-4ae4-a810-5f60086f0703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:43.531235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c160781-9a81-4506-869b-411cff0e61b2 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:43.531509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c160781-9a81-4506-869b-411cff0e61b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:43.584959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-5c160781-9a81-4506-869b-411cff0e61b2 None None] MFA Rules not processed for user `243e592fbf1844d89f55dd5035a684f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:43.589092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5c160781-9a81-4506-869b-411cff0e61b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:43.620806 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 836/1659] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:00:43.632879 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c160781-9a81-4506-869b-411cff0e61b2 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:43.662997 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:43.665417 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c160781-9a81-4506-869b-411cff0e61b2 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:43.695404 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7cfab-06d7-4e2b-b701-70edb1257f63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:43.696304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7cfab-06d7-4e2b-b701-70edb1257f63 service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:43.696755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7cfab-06d7-4e2b-b701-70edb1257f63 service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:43.697418 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7cfab-06d7-4e2b-b701-70edb1257f63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:43.742859 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7cfab-06d7-4e2b-b701-70edb1257f63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:43.744671 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7cfab-06d7-4e2b-b701-70edb1257f63 service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:43.771025 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 837/1660] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:43.889173 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 824/1661] 10.222.0.17 () {58 vars in 1057 bytes} [Thu Jun 16 23:00:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:43.900179 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7cfab-06d7-4e2b-b701-70edb1257f63 service nova] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:43.917545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7cfab-06d7-4e2b-b701-70edb1257f63 service nova] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:43.937262 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210b6b5f-7944-4123-9f3c-b43ffde1c96f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9593939641c4c7797c1981206b40e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:43.938201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b6b5f-7944-4123-9f3c-b43ffde1c96f service placement] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:43.938712 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b6b5f-7944-4123-9f3c-b43ffde1c96f service placement] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:43.939176 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210b6b5f-7944-4123-9f3c-b43ffde1c96f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:43.942875 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-210b6b5f-7944-4123-9f3c-b43ffde1c96f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:43.966446 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210b6b5f-7944-4123-9f3c-b43ffde1c96f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:43.968452 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210b6b5f-7944-4123-9f3c-b43ffde1c96f service placement] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:44.005065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 838/1662] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:44.249714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77443615-62d2-4ad9-aa07-7d715dbc8b2f None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:44.250585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77443615-62d2-4ad9-aa07-7d715dbc8b2f None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:44.250994 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77443615-62d2-4ad9-aa07-7d715dbc8b2f None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:44.253123 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 825/1663] 10.222.0.17 () {54 vars in 961 bytes} [Thu Jun 16 23:00:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 23:00:44.281268 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e0fb4-18cc-4c6e-89af-55419006b8ff None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:44.281776 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e0fb4-18cc-4c6e-89af-55419006b8ff None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:44.281926 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346e0fb4-18cc-4c6e-89af-55419006b8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:44.367429 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-346e0fb4-18cc-4c6e-89af-55419006b8ff None None] MFA Rules not processed for user `4180f2593149437fa27effd8d1338d10`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:44.374756 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-346e0fb4-18cc-4c6e-89af-55419006b8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:44.426630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 839/1664] 10.222.0.17 () {58 vars in 1058 bytes} [Thu Jun 16 23:00:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:44.435539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d51505-bb98-499c-b844-7b323fe5f4a3 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:44.435823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d51505-bb98-499c-b844-7b323fe5f4a3 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:44.436101 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d51505-bb98-499c-b844-7b323fe5f4a3 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:44.443168 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 826/1665] 10.222.0.17 () {54 vars in 912 bytes} [Thu Jun 16 23:00:44 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 23:00:44.446401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f9d4c-62e6-448b-9181-fde23cea24ea None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:44.446870 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f9d4c-62e6-448b-9181-fde23cea24ea None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:44.447534 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8f9d4c-62e6-448b-9181-fde23cea24ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:44.496654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-bc8f9d4c-62e6-448b-9181-fde23cea24ea None None] MFA Rules not processed for user `4180f2593149437fa27effd8d1338d10`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:44.500504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-bc8f9d4c-62e6-448b-9181-fde23cea24ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:44.550526 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 840/1666] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 23:00:44 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:44.652845 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d51505-bb98-499c-b844-7b323fe5f4a3 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:44.690249 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d51505-bb98-499c-b844-7b323fe5f4a3 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:44.704974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef183c-6de5-4e6d-965e-8c0177d186b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:44.705659 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef183c-6de5-4e6d-965e-8c0177d186b5 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:44.705950 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef183c-6de5-4e6d-965e-8c0177d186b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:44.706232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef183c-6de5-4e6d-965e-8c0177d186b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:44.741351 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef183c-6de5-4e6d-965e-8c0177d186b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:44.742964 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef183c-6de5-4e6d-965e-8c0177d186b5 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:44.787064 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 827/1667] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:46.864422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b47d72-8cb4-45fd-8b8a-591017bbb39d None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:46.864969 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b47d72-8cb4-45fd-8b8a-591017bbb39d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:46.865244 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b47d72-8cb4-45fd-8b8a-591017bbb39d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:46.922333 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-94b47d72-8cb4-45fd-8b8a-591017bbb39d None None] MFA Rules not processed for user `9e3a2d16133b4af8a208682de8a7d111`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:46.923832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-94b47d72-8cb4-45fd-8b8a-591017bbb39d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:47.196080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 841/1668] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:47.227282 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef183c-6de5-4e6d-965e-8c0177d186b5 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:47.280607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef183c-6de5-4e6d-965e-8c0177d186b5 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:47.296752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454db135-5530-4d73-b8c6-1ac6996dd5cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:47.297437 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454db135-5530-4d73-b8c6-1ac6996dd5cd service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:47.297887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454db135-5530-4d73-b8c6-1ac6996dd5cd service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:47.298324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454db135-5530-4d73-b8c6-1ac6996dd5cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:47.318765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454db135-5530-4d73-b8c6-1ac6996dd5cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:47.321251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454db135-5530-4d73-b8c6-1ac6996dd5cd service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:47.353397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 828/1669] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:47.454346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c160d-ed2d-48b6-a5b5-4d23cfde1369 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:47.454962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c160d-ed2d-48b6-a5b5-4d23cfde1369 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:47.455570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619c160d-ed2d-48b6-a5b5-4d23cfde1369 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:47.495232 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-619c160d-ed2d-48b6-a5b5-4d23cfde1369 None None] MFA Rules not processed for user `243e592fbf1844d89f55dd5035a684f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:47.511387 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-619c160d-ed2d-48b6-a5b5-4d23cfde1369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:47.545498 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 842/1670] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:47 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:48.278620 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454db135-5530-4d73-b8c6-1ac6996dd5cd service nova] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:48.387971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8798e-646a-4534-a911-74928cdd6de4 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:48.388368 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8798e-646a-4534-a911-74928cdd6de4 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:48.388454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8798e-646a-4534-a911-74928cdd6de4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:48.456189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-30d8798e-646a-4534-a911-74928cdd6de4 None None] MFA Rules not processed for user `faf778e713b54a638c1db4a5db6180b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:48.466200 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-30d8798e-646a-4534-a911-74928cdd6de4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:48.564946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:48.578443 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454db135-5530-4d73-b8c6-1ac6996dd5cd service nova] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:48.598564 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d074a7-75d0-418c-a92f-4fc59712b0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:48.601647 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d074a7-75d0-418c-a92f-4fc59712b0e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:48.601885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d074a7-75d0-418c-a92f-4fc59712b0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:48.603011 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d074a7-75d0-418c-a92f-4fc59712b0e3 admin admin] PATH_INFO: `/v3/users/1b9b9250d43044e5a20ebd0cea2e9c4d` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:48.607004 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d074a7-75d0-418c-a92f-4fc59712b0e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b9b9250d43044e5a20ebd0cea2e9c4d)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:48.612453 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d074a7-75d0-418c-a92f-4fc59712b0e3 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:48.714123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 843/1671] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 23:00:48 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Jun 16 23:00:48.852531 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-38d074a7-75d0-418c-a92f-4fc59712b0e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b9b9250d43044e5a20ebd0cea2e9c4d'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:48.853264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-38d074a7-75d0-418c-a92f-4fc59712b0e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b9b9250d43044e5a20ebd0cea2e9c4d'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:48.869263 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-38d074a7-75d0-418c-a92f-4fc59712b0e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b9b9250d43044e5a20ebd0cea2e9c4d'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:48.878033 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-38d074a7-75d0-418c-a92f-4fc59712b0e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b9b9250d43044e5a20ebd0cea2e9c4d'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:48.932781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-38d074a7-75d0-418c-a92f-4fc59712b0e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b9b9250d43044e5a20ebd0cea2e9c4d'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:48.965849 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 829/1672] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:00:48 2022] DELETE /identity/v3/users/1b9b9250d43044e5a20ebd0cea2e9c4d => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:00:48.970790 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cae2c-7fb5-45d6-b4d3-676435bec331 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:48.971097 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cae2c-7fb5-45d6-b4d3-676435bec331 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:48.971374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329cae2c-7fb5-45d6-b4d3-676435bec331 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:48.988198 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d074a7-75d0-418c-a92f-4fc59712b0e3 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:48.989677 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-38d074a7-75d0-418c-a92f-4fc59712b0e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:49.031631 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-329cae2c-7fb5-45d6-b4d3-676435bec331 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:49.036985 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-329cae2c-7fb5-45d6-b4d3-676435bec331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:49.056871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:49.218482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 844/1673] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:49.264550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d074a7-75d0-418c-a92f-4fc59712b0e3 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:49.280100 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff5bc9a-1b17-4092-a524-2ab5f7e1438c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef518c9430e4274b1066241f821f4a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:49.280861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5bc9a-1b17-4092-a524-2ab5f7e1438c service swift] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:49.281225 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5bc9a-1b17-4092-a524-2ab5f7e1438c service swift] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:49.281843 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff5bc9a-1b17-4092-a524-2ab5f7e1438c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:49.287715 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-2ff5bc9a-1b17-4092-a524-2ab5f7e1438c service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:49.306519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cae2c-7fb5-45d6-b4d3-676435bec331 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:49.310036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-329cae2c-7fb5-45d6-b4d3-676435bec331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:49.460381 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329cae2c-7fb5-45d6-b4d3-676435bec331 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:49.467818 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e803dd-2008-4894-b213-83b97649a0bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:49.468837 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e803dd-2008-4894-b213-83b97649a0bf service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:49.469330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e803dd-2008-4894-b213-83b97649a0bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:49.469868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e803dd-2008-4894-b213-83b97649a0bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:49.483489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e803dd-2008-4894-b213-83b97649a0bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:49.485674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e803dd-2008-4894-b213-83b97649a0bf service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:49.498821 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 845/1674] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:49.572231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5bc9a-1b17-4092-a524-2ab5f7e1438c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:49.582972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff5bc9a-1b17-4092-a524-2ab5f7e1438c service swift] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:49.633435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 830/1675] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 23:00:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 646 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 23:00:51.044645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e803dd-2008-4894-b213-83b97649a0bf service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:51.046398 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-29e803dd-2008-4894-b213-83b97649a0bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:51.085363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e803dd-2008-4894-b213-83b97649a0bf service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:51.097313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7ee44d-2241-4940-bed2-f9ae97f47312 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:51.100209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ee44d-2241-4940-bed2-f9ae97f47312 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:51.101153 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ee44d-2241-4940-bed2-f9ae97f47312 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:51.102664 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7ee44d-2241-4940-bed2-f9ae97f47312 admin admin] PATH_INFO: `/v3/users/976e8af4b4b34ad89bbc073ba808eeaa` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:51.111520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ee44d-2241-4940-bed2-f9ae97f47312 admin admin] RBAC: Authorizing `identity:delete_user(user_id=976e8af4b4b34ad89bbc073ba808eeaa)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:51.118189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: WARNING py.warnings [None req-9d7ee44d-2241-4940-bed2-f9ae97f47312 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 23:00:51.118189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: warnings.warn(msg) Jun 16 23:00:51.118189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: Jun 16 23:00:51.123334 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d7ee44d-2241-4940-bed2-f9ae97f47312 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:51.397565 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9d7ee44d-2241-4940-bed2-f9ae97f47312 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '976e8af4b4b34ad89bbc073ba808eeaa'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:51.441393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9d7ee44d-2241-4940-bed2-f9ae97f47312 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '976e8af4b4b34ad89bbc073ba808eeaa'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:51.456223 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9d7ee44d-2241-4940-bed2-f9ae97f47312 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '976e8af4b4b34ad89bbc073ba808eeaa'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:51.555657 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:51.560585 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9d7ee44d-2241-4940-bed2-f9ae97f47312 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '976e8af4b4b34ad89bbc073ba808eeaa'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:51.563727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9d7ee44d-2241-4940-bed2-f9ae97f47312 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '976e8af4b4b34ad89bbc073ba808eeaa'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:51.605679 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 846/1676] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:00:51 2022] DELETE /identity/v3/users/976e8af4b4b34ad89bbc073ba808eeaa => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:00:51.608274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b50ec-9e92-4476-8e98-3ca321f304bd None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:51.608754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b50ec-9e92-4476-8e98-3ca321f304bd None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:51.608900 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61b50ec-9e92-4476-8e98-3ca321f304bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:51.665828 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-e61b50ec-9e92-4476-8e98-3ca321f304bd None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:51.671292 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-e61b50ec-9e92-4476-8e98-3ca321f304bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:51.868764 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 831/1677] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:51.915449 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a9f0e-c8d9-40d4-b5aa-e122f5ad7579 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:51.915799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a9f0e-c8d9-40d4-b5aa-e122f5ad7579 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:51.916088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a9f0e-c8d9-40d4-b5aa-e122f5ad7579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:51.972952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b50ec-9e92-4476-8e98-3ca321f304bd None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:51.975619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-e61b50ec-9e92-4476-8e98-3ca321f304bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:52.002146 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-2a4a9f0e-c8d9-40d4-b5aa-e122f5ad7579 None None] MFA Rules not processed for user `a1b1a557a3834b98a3397c41665b5dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:52.006220 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-2a4a9f0e-c8d9-40d4-b5aa-e122f5ad7579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:52.177832 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61b50ec-9e92-4476-8e98-3ca321f304bd None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:52.197363 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28d3d90-7582-4afb-bbe8-605d9af3091d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:52.200422 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d3d90-7582-4afb-bbe8-605d9af3091d service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:52.203038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d3d90-7582-4afb-bbe8-605d9af3091d service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:52.204759 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28d3d90-7582-4afb-bbe8-605d9af3091d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:52.230066 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d3d90-7582-4afb-bbe8-605d9af3091d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:52.233589 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28d3d90-7582-4afb-bbe8-605d9af3091d service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:52.245198 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 847/1678] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:52.256138 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 832/1679] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:00:52.370801 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a9f0e-c8d9-40d4-b5aa-e122f5ad7579 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:52.372527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-2a4a9f0e-c8d9-40d4-b5aa-e122f5ad7579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:52.572654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a9f0e-c8d9-40d4-b5aa-e122f5ad7579 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:52.596044 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42d1bc-afcd-4bc2-affb-635eb2c5f0d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:52.598092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42d1bc-afcd-4bc2-affb-635eb2c5f0d2 service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:52.599549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42d1bc-afcd-4bc2-affb-635eb2c5f0d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:52.600594 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d42d1bc-afcd-4bc2-affb-635eb2c5f0d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:52.640548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42d1bc-afcd-4bc2-affb-635eb2c5f0d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:52.642742 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d42d1bc-afcd-4bc2-affb-635eb2c5f0d2 service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:52.686748 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 848/1680] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:53.828577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c5aa9-3f0b-4f40-9e63-59d5c030b533 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:53.830203 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c5aa9-3f0b-4f40-9e63-59d5c030b533 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:53.830835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c5aa9-3f0b-4f40-9e63-59d5c030b533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:53.863064 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:53.935915 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-633c5aa9-3f0b-4f40-9e63-59d5c030b533 None None] MFA Rules not processed for user `6b794f7aa2a7488e91f7d9ea35a45d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:53.940434 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-633c5aa9-3f0b-4f40-9e63-59d5c030b533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:53.976045 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 833/1681] 10.222.0.26 () {58 vars in 1015 bytes} [Thu Jun 16 23:00:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:00:54.401045 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42d1bc-afcd-4bc2-affb-635eb2c5f0d2 service nova] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:54.402930 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-9d42d1bc-afcd-4bc2-affb-635eb2c5f0d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:54.428076 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d42d1bc-afcd-4bc2-affb-635eb2c5f0d2 service nova] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:54.438986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec398c-0440-47c6-afab-22ead9fbe7ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:54.439959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec398c-0440-47c6-afab-22ead9fbe7ed service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:54.441473 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec398c-0440-47c6-afab-22ead9fbe7ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:54.441783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eec398c-0440-47c6-afab-22ead9fbe7ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:54.464641 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec398c-0440-47c6-afab-22ead9fbe7ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:54.470729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eec398c-0440-47c6-afab-22ead9fbe7ed service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:54.485549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 849/1682] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:00:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:00:54.691253 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ba6e7-aaa5-487a-8aa8-7e96d9830ce3 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:54.691621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ba6e7-aaa5-487a-8aa8-7e96d9830ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:54.691979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ba6e7-aaa5-487a-8aa8-7e96d9830ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:54.730124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-336ba6e7-aaa5-487a-8aa8-7e96d9830ce3 None None] MFA Rules not processed for user `6a993f5d14b64fccad723abecfcadf90`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:00:54.732026 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-336ba6e7-aaa5-487a-8aa8-7e96d9830ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:54.925754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 834/1683] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:00:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:00:55.022818 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec398c-0440-47c6-afab-22ead9fbe7ed service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:55.024770 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-2eec398c-0440-47c6-afab-22ead9fbe7ed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:55.046666 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ba6e7-aaa5-487a-8aa8-7e96d9830ce3 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:55.048469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-336ba6e7-aaa5-487a-8aa8-7e96d9830ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:55.051791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eec398c-0440-47c6-afab-22ead9fbe7ed service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:55.064047 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d55bc9e-2ed8-4e67-be49-9943c3642da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:55.065712 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55bc9e-2ed8-4e67-be49-9943c3642da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:55.070052 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55bc9e-2ed8-4e67-be49-9943c3642da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:55.070503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d55bc9e-2ed8-4e67-be49-9943c3642da9 admin admin] PATH_INFO: `/v3/projects/f3cf903d4bf142e5909f9ea88d639a47` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:55.075229 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d55bc9e-2ed8-4e67-be49-9943c3642da9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3cf903d4bf142e5909f9ea88d639a47)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:55.077080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d55bc9e-2ed8-4e67-be49-9943c3642da9 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:55.092060 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ba6e7-aaa5-487a-8aa8-7e96d9830ce3 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:55.124745 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1d259-8462-48ea-afd6-b7c32ffa121d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:55.129297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1d259-8462-48ea-afd6-b7c32ffa121d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:55.130690 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1d259-8462-48ea-afd6-b7c32ffa121d admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:55.131572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c1d259-8462-48ea-afd6-b7c32ffa121d admin admin] PATH_INFO: `/v3/projects/a94ff92482e8490e85ee579c2d59d611` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:55.142707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1d259-8462-48ea-afd6-b7c32ffa121d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a94ff92482e8490e85ee579c2d59d611)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:55.146861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: WARNING py.warnings [None req-89c1d259-8462-48ea-afd6-b7c32ffa121d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 23:00:55.146861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: warnings.warn(msg) Jun 16 23:00:55.146861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: Jun 16 23:00:55.151035 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c1d259-8462-48ea-afd6-b7c32ffa121d admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:55.233622 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:55.243449 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8d55bc9e-2ed8-4e67-be49-9943c3642da9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3cf903d4bf142e5909f9ea88d639a47', 'request_id': 'req-8d55bc9e-2ed8-4e67-be49-9943c3642da9'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:55.266710 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8d55bc9e-2ed8-4e67-be49-9943c3642da9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3cf903d4bf142e5909f9ea88d639a47', 'request_id': 'req-8d55bc9e-2ed8-4e67-be49-9943c3642da9'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:55.310270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8d55bc9e-2ed8-4e67-be49-9943c3642da9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3cf903d4bf142e5909f9ea88d639a47', 'request_id': 'req-8d55bc9e-2ed8-4e67-be49-9943c3642da9'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:55.336886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8d55bc9e-2ed8-4e67-be49-9943c3642da9 admin admin] The token cache is being invalidate because project f3cf903d4bf142e5909f9ea88d639a47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:00:55.338515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8d55bc9e-2ed8-4e67-be49-9943c3642da9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:55.340106 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8d55bc9e-2ed8-4e67-be49-9943c3642da9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:55.347914 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 850/1684] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:00:55 2022] DELETE /identity/v3/projects/f3cf903d4bf142e5909f9ea88d639a47 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:00:55.352457 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d55bc9e-2ed8-4e67-be49-9943c3642da9 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:55.355795 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-8d55bc9e-2ed8-4e67-be49-9943c3642da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:55.474058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-89c1d259-8462-48ea-afd6-b7c32ffa121d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a94ff92482e8490e85ee579c2d59d611', 'request_id': 'req-89c1d259-8462-48ea-afd6-b7c32ffa121d'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:55.542128 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-89c1d259-8462-48ea-afd6-b7c32ffa121d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a94ff92482e8490e85ee579c2d59d611', 'request_id': 'req-89c1d259-8462-48ea-afd6-b7c32ffa121d'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:55.574489 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-89c1d259-8462-48ea-afd6-b7c32ffa121d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a94ff92482e8490e85ee579c2d59d611', 'request_id': 'req-89c1d259-8462-48ea-afd6-b7c32ffa121d'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:55.587063 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d55bc9e-2ed8-4e67-be49-9943c3642da9 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:55.609403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-89c1d259-8462-48ea-afd6-b7c32ffa121d admin admin] The token cache is being invalidate because project a94ff92482e8490e85ee579c2d59d611 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:00:55.609403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-89c1d259-8462-48ea-afd6-b7c32ffa121d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:55.612633 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-89c1d259-8462-48ea-afd6-b7c32ffa121d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:55.616923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 835/1685] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:00:55 2022] DELETE /identity/v3/projects/a94ff92482e8490e85ee579c2d59d611 => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:00:55.621710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1d259-8462-48ea-afd6-b7c32ffa121d admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:00:55.623874 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-89c1d259-8462-48ea-afd6-b7c32ffa121d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:00:55.846202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c1d259-8462-48ea-afd6-b7c32ffa121d admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:00:55.867623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ad11c-8d1a-485f-881b-7aee6e3c1bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:55.868541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ad11c-8d1a-485f-881b-7aee6e3c1bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:55.868833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ad11c-8d1a-485f-881b-7aee6e3c1bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:55.869119 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8ad11c-8d1a-485f-881b-7aee6e3c1bac admin admin] PATH_INFO: `/v3/users/a344653f0f0946038ef37ae2c16ab779` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:55.875488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ad11c-8d1a-485f-881b-7aee6e3c1bac admin admin] RBAC: Authorizing `identity:delete_user(user_id=a344653f0f0946038ef37ae2c16ab779)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:55.883676 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8ad11c-8d1a-485f-881b-7aee6e3c1bac admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:55.980954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:55.988476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec34808-6578-4c22-abaa-83ea00ab1ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:00:56.000551 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec34808-6578-4c22-abaa-83ea00ab1ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:00:56.001021 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec34808-6578-4c22-abaa-83ea00ab1ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:00:56.001219 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec34808-6578-4c22-abaa-83ea00ab1ef6 admin admin] PATH_INFO: `/v3/users/8b6b04b804df45f897aade6dfc76595e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:00:56.013340 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec34808-6578-4c22-abaa-83ea00ab1ef6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b6b04b804df45f897aade6dfc76595e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:00:56.015386 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec34808-6578-4c22-abaa-83ea00ab1ef6 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:00:56.082090 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5a8ad11c-8d1a-485f-881b-7aee6e3c1bac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a344653f0f0946038ef37ae2c16ab779'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:56.082913 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5a8ad11c-8d1a-485f-881b-7aee6e3c1bac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a344653f0f0946038ef37ae2c16ab779'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:56.108588 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5a8ad11c-8d1a-485f-881b-7aee6e3c1bac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a344653f0f0946038ef37ae2c16ab779'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:56.111606 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5a8ad11c-8d1a-485f-881b-7aee6e3c1bac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a344653f0f0946038ef37ae2c16ab779'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:56.148144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5a8ad11c-8d1a-485f-881b-7aee6e3c1bac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a344653f0f0946038ef37ae2c16ab779'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:56.187181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8ec34808-6578-4c22-abaa-83ea00ab1ef6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b6b04b804df45f897aade6dfc76595e'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:56.202707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 836/1686] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:00:55 2022] DELETE /identity/v3/users/a344653f0f0946038ef37ae2c16ab779 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:00:56.215351 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8ec34808-6578-4c22-abaa-83ea00ab1ef6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b6b04b804df45f897aade6dfc76595e'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:56.216783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8ec34808-6578-4c22-abaa-83ea00ab1ef6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b6b04b804df45f897aade6dfc76595e'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:56.271851 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8ec34808-6578-4c22-abaa-83ea00ab1ef6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b6b04b804df45f897aade6dfc76595e'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:56.272629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8ec34808-6578-4c22-abaa-83ea00ab1ef6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b6b04b804df45f897aade6dfc76595e'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:00:56.301448 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:00:56.316778 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 851/1687] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:00:55 2022] DELETE /identity/v3/users/8b6b04b804df45f897aade6dfc76595e => generated 0 bytes in 965 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:01:00.940634 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ad11c-8d1a-485f-881b-7aee6e3c1bac admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:00.944537 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-5a8ad11c-8d1a-485f-881b-7aee6e3c1bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:01.161498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8ad11c-8d1a-485f-881b-7aee6e3c1bac admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:01.180476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddebe15-b4f4-483e-b3b8-bf243e9b3719 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:01.180967 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddebe15-b4f4-483e-b3b8-bf243e9b3719 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:01.181047 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddebe15-b4f4-483e-b3b8-bf243e9b3719 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:01.182584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 852/1688] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 23:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 23:01:01.344034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f1eba7-4642-43b8-bd54-fa2b7b8e6781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:01.344931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1eba7-4642-43b8-bd54-fa2b7b8e6781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:01.345214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1eba7-4642-43b8-bd54-fa2b7b8e6781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:01.345490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f1eba7-4642-43b8-bd54-fa2b7b8e6781 admin admin] PATH_INFO: `/v3/projects/db045edf3677494ab6117428fd013c73` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:01.354022 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1eba7-4642-43b8-bd54-fa2b7b8e6781 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db045edf3677494ab6117428fd013c73)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:01.365959 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f1eba7-4642-43b8-bd54-fa2b7b8e6781 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:01.421951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c9622-27bc-4edd-8a2c-490f2252c628 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:01.422301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c9622-27bc-4edd-8a2c-490f2252c628 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:01.422564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-916c9622-27bc-4edd-8a2c-490f2252c628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:01.477560 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-916c9622-27bc-4edd-8a2c-490f2252c628 None None] MFA Rules not processed for user `4180f2593149437fa27effd8d1338d10`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:01.480932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-916c9622-27bc-4edd-8a2c-490f2252c628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:01.569532 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-85f1eba7-4642-43b8-bd54-fa2b7b8e6781 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db045edf3677494ab6117428fd013c73', 'request_id': 'req-85f1eba7-4642-43b8-bd54-fa2b7b8e6781'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:01.574915 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:01.586034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-85f1eba7-4642-43b8-bd54-fa2b7b8e6781 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db045edf3677494ab6117428fd013c73', 'request_id': 'req-85f1eba7-4642-43b8-bd54-fa2b7b8e6781'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:01.629358 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-85f1eba7-4642-43b8-bd54-fa2b7b8e6781 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db045edf3677494ab6117428fd013c73', 'request_id': 'req-85f1eba7-4642-43b8-bd54-fa2b7b8e6781'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:01.668223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-85f1eba7-4642-43b8-bd54-fa2b7b8e6781 admin admin] The token cache is being invalidate because project db045edf3677494ab6117428fd013c73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:01:01.668514 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-85f1eba7-4642-43b8-bd54-fa2b7b8e6781 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:01.674282 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-85f1eba7-4642-43b8-bd54-fa2b7b8e6781 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:01.676924 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 837/1689] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:01:00 2022] DELETE /identity/v3/projects/db045edf3677494ab6117428fd013c73 => generated 0 bytes in 737 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:01.687061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c818-5cb0-4c4e-95a8-5d360890b7b9 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:01.688660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c818-5cb0-4c4e-95a8-5d360890b7b9 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:01.692141 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d01c818-5cb0-4c4e-95a8-5d360890b7b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:01.716062 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 853/1690] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 23:01:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:01.746922 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-1d01c818-5cb0-4c4e-95a8-5d360890b7b9 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:01.750681 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-1d01c818-5cb0-4c4e-95a8-5d360890b7b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:01.795782 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c9622-27bc-4edd-8a2c-490f2252c628 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:01.797536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-916c9622-27bc-4edd-8a2c-490f2252c628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:01.843344 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-916c9622-27bc-4edd-8a2c-490f2252c628 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:01.962922 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 838/1691] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:01:01.977889 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af02307-ec32-4959-bf11-85939bb8854d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:01.980297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af02307-ec32-4959-bf11-85939bb8854d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:01.981570 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af02307-ec32-4959-bf11-85939bb8854d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:02.123623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-5af02307-ec32-4959-bf11-85939bb8854d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:02.131927 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-5af02307-ec32-4959-bf11-85939bb8854d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:02.137515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afdbda1-2dc8-4743-a6cd-c1bb9c04e347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:02.139793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdbda1-2dc8-4743-a6cd-c1bb9c04e347 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:02.141439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdbda1-2dc8-4743-a6cd-c1bb9c04e347 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:02.142737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afdbda1-2dc8-4743-a6cd-c1bb9c04e347 admin admin] PATH_INFO: `/v3/projects/6d06ad8f0ca44a6db60ba1ce6b0450b9` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:02.144690 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdbda1-2dc8-4743-a6cd-c1bb9c04e347 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d06ad8f0ca44a6db60ba1ce6b0450b9)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:02.150318 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afdbda1-2dc8-4743-a6cd-c1bb9c04e347 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:02.164061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 839/1692] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:01:02.170019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af02307-ec32-4959-bf11-85939bb8854d None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:02.197857 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af02307-ec32-4959-bf11-85939bb8854d None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:02.215727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9c535-3001-40d1-a5ec-17af6ab6ce44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:02.216703 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9c535-3001-40d1-a5ec-17af6ab6ce44 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:02.217137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9c535-3001-40d1-a5ec-17af6ab6ce44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:02.217567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b9c535-3001-40d1-a5ec-17af6ab6ce44 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:02.220113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9c535-3001-40d1-a5ec-17af6ab6ce44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:01:02.220921 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9c535-3001-40d1-a5ec-17af6ab6ce44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:02.229653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b9c535-3001-40d1-a5ec-17af6ab6ce44 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:02.283457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 840/1693] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:01:02.289392 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9c535-3001-40d1-a5ec-17af6ab6ce44 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:02.318735 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b9c535-3001-40d1-a5ec-17af6ab6ce44 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:02.355886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1afdbda1-2dc8-4743-a6cd-c1bb9c04e347 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d06ad8f0ca44a6db60ba1ce6b0450b9', 'request_id': 'req-1afdbda1-2dc8-4743-a6cd-c1bb9c04e347'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:02.390470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1afdbda1-2dc8-4743-a6cd-c1bb9c04e347 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d06ad8f0ca44a6db60ba1ce6b0450b9', 'request_id': 'req-1afdbda1-2dc8-4743-a6cd-c1bb9c04e347'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:02.428234 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1afdbda1-2dc8-4743-a6cd-c1bb9c04e347 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d06ad8f0ca44a6db60ba1ce6b0450b9', 'request_id': 'req-1afdbda1-2dc8-4743-a6cd-c1bb9c04e347'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:02.478008 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1afdbda1-2dc8-4743-a6cd-c1bb9c04e347 admin admin] The token cache is being invalidate because project 6d06ad8f0ca44a6db60ba1ce6b0450b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:01:02.478325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1afdbda1-2dc8-4743-a6cd-c1bb9c04e347 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:02.490899 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1afdbda1-2dc8-4743-a6cd-c1bb9c04e347 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:02.493617 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 854/1694] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:01:01 2022] DELETE /identity/v3/projects/6d06ad8f0ca44a6db60ba1ce6b0450b9 => generated 0 bytes in 695 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:02.498522 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11a671-ab2e-40f8-b87e-521bf9e49674 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:02.498859 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11a671-ab2e-40f8-b87e-521bf9e49674 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:02.505426 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16596cd2-f841-4c99-ac7d-fe9541ac4ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:02.506052 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da11a671-ab2e-40f8-b87e-521bf9e49674 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:02.506169 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16596cd2-f841-4c99-ac7d-fe9541ac4ccb admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:02.506417 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16596cd2-f841-4c99-ac7d-fe9541ac4ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:02.507358 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16596cd2-f841-4c99-ac7d-fe9541ac4ccb admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:02.508215 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16596cd2-f841-4c99-ac7d-fe9541ac4ccb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:02.555408 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-da11a671-ab2e-40f8-b87e-521bf9e49674 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:02.559830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16596cd2-f841-4c99-ac7d-fe9541ac4ccb admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:02.561768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-da11a671-ab2e-40f8-b87e-521bf9e49674 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:02.582074 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 855/1695] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:02.591187 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229de7e2-f002-4c0c-8703-519c70c915f1 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:02.591487 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229de7e2-f002-4c0c-8703-519c70c915f1 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:02.591773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-229de7e2-f002-4c0c-8703-519c70c915f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:02.645636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-229de7e2-f002-4c0c-8703-519c70c915f1 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:02.647734 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-229de7e2-f002-4c0c-8703-519c70c915f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:02.675590 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 856/1696] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:02.683473 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229de7e2-f002-4c0c-8703-519c70c915f1 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:02.689548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 841/1697] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:01:02 2022] POST /identity/v3/projects => generated 391 bytes in 401 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:02.690489 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16596cd2-f841-4c99-ac7d-fe9541ac4ccb admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:02.692779 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-16596cd2-f841-4c99-ac7d-fe9541ac4ccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:02.716059 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-229de7e2-f002-4c0c-8703-519c70c915f1 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:02.730358 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:02.881062 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:02.911980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16596cd2-f841-4c99-ac7d-fe9541ac4ccb admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:02.922807 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d5b41-0947-4d97-a590-7ccc70a1dbff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:02.923718 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d5b41-0947-4d97-a590-7ccc70a1dbff admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:02.924033 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d5b41-0947-4d97-a590-7ccc70a1dbff admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:02.924305 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5d5b41-0947-4d97-a590-7ccc70a1dbff admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:02.936828 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cd371-702b-4796-8593-421d0a8532c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:02.939043 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d5b41-0947-4d97-a590-7ccc70a1dbff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:02.940078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cd371-702b-4796-8593-421d0a8532c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:02.940721 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cd371-702b-4796-8593-421d0a8532c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:02.941084 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5d5b41-0947-4d97-a590-7ccc70a1dbff admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:02.942472 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722cd371-702b-4796-8593-421d0a8532c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:02.944125 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cd371-702b-4796-8593-421d0a8532c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:01:02.960798 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cd371-702b-4796-8593-421d0a8532c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:02.963391 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722cd371-702b-4796-8593-421d0a8532c2 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:03.014951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 857/1698] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:01:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:01:03.015121 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 842/1699] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:01:02 2022] POST /identity/v3/users => generated 296 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:03.018690 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d5b41-0947-4d97-a590-7ccc70a1dbff admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:03.020542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cd371-702b-4796-8593-421d0a8532c2 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:03.045514 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5d5b41-0947-4d97-a590-7ccc70a1dbff admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:03.067837 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6895961-42a0-4802-aca7-e9827300fb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:03.069073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6895961-42a0-4802-aca7-e9827300fb7f admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:03.069573 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6895961-42a0-4802-aca7-e9827300fb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:03.073338 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6895961-42a0-4802-aca7-e9827300fb7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:03.074891 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6895961-42a0-4802-aca7-e9827300fb7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:03.077455 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6895961-42a0-4802-aca7-e9827300fb7f admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:03.077601 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722cd371-702b-4796-8593-421d0a8532c2 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:03.099131 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47805f5-f6d6-4fd1-a76f-fe7a92adb11e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:03.099844 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47805f5-f6d6-4fd1-a76f-fe7a92adb11e admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:03.100137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47805f5-f6d6-4fd1-a76f-fe7a92adb11e admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:03.100426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47805f5-f6d6-4fd1-a76f-fe7a92adb11e admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:03.101178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47805f5-f6d6-4fd1-a76f-fe7a92adb11e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:03.126832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47805f5-f6d6-4fd1-a76f-fe7a92adb11e admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:03.148518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 858/1700] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:03.149918 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47805f5-f6d6-4fd1-a76f-fe7a92adb11e admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:03.158271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-d47805f5-f6d6-4fd1-a76f-fe7a92adb11e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:03.174177 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 843/1701] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:01:03 2022] POST /identity/v3/projects => generated 377 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:03.174177 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6895961-42a0-4802-aca7-e9827300fb7f admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:03.432985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6895961-42a0-4802-aca7-e9827300fb7f admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:03.440333 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47805f5-f6d6-4fd1-a76f-fe7a92adb11e admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:03.482465 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3852d51-13bf-4285-a35d-862cefa4c4f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:03.486468 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3852d51-13bf-4285-a35d-862cefa4c4f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:03.491901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3852d51-13bf-4285-a35d-862cefa4c4f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:03.492663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3852d51-13bf-4285-a35d-862cefa4c4f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:03.496951 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3852d51-13bf-4285-a35d-862cefa4c4f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:03.501751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3852d51-13bf-4285-a35d-862cefa4c4f8 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:03.549661 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 844/1702] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:03.568423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3852d51-13bf-4285-a35d-862cefa4c4f8 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:03.595170 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f3088-419e-4011-9e92-5b4a6f7fe9ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:03.596799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f3088-419e-4011-9e92-5b4a6f7fe9ee service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:03.597337 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f3088-419e-4011-9e92-5b4a6f7fe9ee service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:03.598364 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f3088-419e-4011-9e92-5b4a6f7fe9ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:03.605378 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-388f3088-419e-4011-9e92-5b4a6f7fe9ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:03.610277 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3852d51-13bf-4285-a35d-862cefa4c4f8 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:03.623095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1311dfd-9b0b-46cd-addf-75464761aac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:03.624089 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1311dfd-9b0b-46cd-addf-75464761aac0 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:03.624379 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1311dfd-9b0b-46cd-addf-75464761aac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:03.624836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1311dfd-9b0b-46cd-addf-75464761aac0 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:03.630318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1311dfd-9b0b-46cd-addf-75464761aac0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:03.632494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1311dfd-9b0b-46cd-addf-75464761aac0 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:03.634331 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f3088-419e-4011-9e92-5b4a6f7fe9ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:03.642285 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f3088-419e-4011-9e92-5b4a6f7fe9ee service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:03.691049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 859/1703] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 542 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:03.692849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f3088-419e-4011-9e92-5b4a6f7fe9ee service nova] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:03.713457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 845/1704] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:01:03 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:03.718325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1311dfd-9b0b-46cd-addf-75464761aac0 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:03.727524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f3088-419e-4011-9e92-5b4a6f7fe9ee service nova] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:03.733337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1311dfd-9b0b-46cd-addf-75464761aac0 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:03.740824 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3da82-a030-41a4-9fee-54591dfb50a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:03.741621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3da82-a030-41a4-9fee-54591dfb50a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:03.742086 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3da82-a030-41a4-9fee-54591dfb50a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:03.742563 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f3da82-a030-41a4-9fee-54591dfb50a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:03.743420 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3da82-a030-41a4-9fee-54591dfb50a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:03.746840 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f3da82-a030-41a4-9fee-54591dfb50a3 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:03.747492 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672a09c0-0134-43fd-9d05-25acc6881fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:03.748662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a09c0-0134-43fd-9d05-25acc6881fcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:03.748764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a09c0-0134-43fd-9d05-25acc6881fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:03.748994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672a09c0-0134-43fd-9d05-25acc6881fcd admin admin] PATH_INFO: `/v3/projects/dec4af79ecf54d3bad6c9aea58ba159f/users/5b21f9e2b5434b00b24f0aeb046f6e50/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:03.766494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 846/1705] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:01:03.785434 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3da82-a030-41a4-9fee-54591dfb50a3 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:03.799870 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:03.809899 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a09c0-0134-43fd-9d05-25acc6881fcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dec4af79ecf54d3bad6c9aea58ba159f, user_id=5b21f9e2b5434b00b24f0aeb046f6e50, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:03.812662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672a09c0-0134-43fd-9d05-25acc6881fcd admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:03.824442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f3da82-a030-41a4-9fee-54591dfb50a3 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:03.847710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44c707-019b-4d76-a797-252fb33f48bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:03.853126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44c707-019b-4d76-a797-252fb33f48bf admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:03.854185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44c707-019b-4d76-a797-252fb33f48bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:03.854642 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce44c707-019b-4d76-a797-252fb33f48bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:03.859861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44c707-019b-4d76-a797-252fb33f48bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:03.883408 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 860/1706] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:03 2022] PUT /identity/v3/projects/dec4af79ecf54d3bad6c9aea58ba159f/users/5b21f9e2b5434b00b24f0aeb046f6e50/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:03.885175 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c51e8-2c77-40af-8997-668943296e77 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:03.885585 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c51e8-2c77-40af-8997-668943296e77 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:03.885872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547c51e8-2c77-40af-8997-668943296e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:03.899376 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce44c707-019b-4d76-a797-252fb33f48bf admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:03.944902 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-547c51e8-2c77-40af-8997-668943296e77 None None] MFA Rules not processed for user `5b21f9e2b5434b00b24f0aeb046f6e50`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:03.946108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-547c51e8-2c77-40af-8997-668943296e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:03.976519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 847/1707] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:03 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:03.994491 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44c707-019b-4d76-a797-252fb33f48bf admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:04.050668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce44c707-019b-4d76-a797-252fb33f48bf admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:04.171065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 861/1708] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:04.175227 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c51e8-2c77-40af-8997-668943296e77 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:04.351178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547c51e8-2c77-40af-8997-668943296e77 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:04.365887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa5864-65a3-4e5d-b080-667e48dae6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:04.367167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa5864-65a3-4e5d-b080-667e48dae6b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:04.367756 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa5864-65a3-4e5d-b080-667e48dae6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:04.368238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa5864-65a3-4e5d-b080-667e48dae6b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:04.369196 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa5864-65a3-4e5d-b080-667e48dae6b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:04.371020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec65e3c-ff7f-4a9e-9313-955a042bd19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:04.372055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec65e3c-ff7f-4a9e-9313-955a042bd19b admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:04.372343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec65e3c-ff7f-4a9e-9313-955a042bd19b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:04.372809 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec65e3c-ff7f-4a9e-9313-955a042bd19b admin admin] PATH_INFO: `/v3/projects/bd58c1c77685485ca8fd862e03dbbd89/users/58ab5db75b234a1fbcc0a5a92973818b/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:04.387618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2aa5864-65a3-4e5d-b080-667e48dae6b0 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:04.404579 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec65e3c-ff7f-4a9e-9313-955a042bd19b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd58c1c77685485ca8fd862e03dbbd89, user_id=58ab5db75b234a1fbcc0a5a92973818b, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:04.407472 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec65e3c-ff7f-4a9e-9313-955a042bd19b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:04.410135 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:04.438542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 848/1709] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:03 2022] PUT /identity/v3/projects/bd58c1c77685485ca8fd862e03dbbd89/users/58ab5db75b234a1fbcc0a5a92973818b/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:04.456533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e68153-7bde-40e0-b02e-cc1b9d5c9e09 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:04.458179 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e68153-7bde-40e0-b02e-cc1b9d5c9e09 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:04.458531 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e68153-7bde-40e0-b02e-cc1b9d5c9e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:04.496991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 862/1710] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:01:04 2022] POST /identity/v3/projects => generated 393 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:04.501268 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa5864-65a3-4e5d-b080-667e48dae6b0 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:04.542583 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-f0e68153-7bde-40e0-b02e-cc1b9d5c9e09 None None] MFA Rules not processed for user `58ab5db75b234a1fbcc0a5a92973818b`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:04.545353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f0e68153-7bde-40e0-b02e-cc1b9d5c9e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:04.663297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 849/1711] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:01:04.669252 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e68153-7bde-40e0-b02e-cc1b9d5c9e09 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:04.709717 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa5864-65a3-4e5d-b080-667e48dae6b0 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:04.733444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f4b4d-5140-4005-a57c-6066d6e7937a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:04.734928 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f4b4d-5140-4005-a57c-6066d6e7937a admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:04.735746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f4b4d-5140-4005-a57c-6066d6e7937a admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:04.736852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f4b4d-5140-4005-a57c-6066d6e7937a admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:04.742516 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f4b4d-5140-4005-a57c-6066d6e7937a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:04.844281 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73f4b4d-5140-4005-a57c-6066d6e7937a admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:04.884936 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e68153-7bde-40e0-b02e-cc1b9d5c9e09 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:04.907264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:04.914476 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e21877-d2a5-4a0a-85bf-bb391209d192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:04.924023 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e21877-d2a5-4a0a-85bf-bb391209d192 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:04.925661 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e21877-d2a5-4a0a-85bf-bb391209d192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:04.926939 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e21877-d2a5-4a0a-85bf-bb391209d192 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:04.928317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e21877-d2a5-4a0a-85bf-bb391209d192 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:04.935680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e21877-d2a5-4a0a-85bf-bb391209d192 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:04.950453 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 863/1712] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:01:04 2022] POST /identity/v3/users => generated 303 bytes in 449 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:04.956552 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978610c-b8b1-45a9-bf7d-cb08e72d65aa None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:04.961459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978610c-b8b1-45a9-bf7d-cb08e72d65aa None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:04.963168 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2978610c-b8b1-45a9-bf7d-cb08e72d65aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:05.022187 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-2978610c-b8b1-45a9-bf7d-cb08e72d65aa None None] MFA Rules not processed for user `9e3a2d16133b4af8a208682de8a7d111`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:05.026104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-2978610c-b8b1-45a9-bf7d-cb08e72d65aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:05.036338 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 850/1713] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:01:04 2022] POST /identity/v3/projects => generated 375 bytes in 365 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:05.037163 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e21877-d2a5-4a0a-85bf-bb391209d192 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:05.182755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e21877-d2a5-4a0a-85bf-bb391209d192 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:05.191897 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0c7a1a-858c-49ba-9cfa-d41542fad01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:05.193207 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c7a1a-858c-49ba-9cfa-d41542fad01b admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:05.197009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c7a1a-858c-49ba-9cfa-d41542fad01b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:05.197597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0c7a1a-858c-49ba-9cfa-d41542fad01b admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:05.199046 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c7a1a-858c-49ba-9cfa-d41542fad01b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:05.210435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0c7a1a-858c-49ba-9cfa-d41542fad01b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:05.217982 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 864/1714] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:05.219632 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978610c-b8b1-45a9-bf7d-cb08e72d65aa None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:05.232140 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2978610c-b8b1-45a9-bf7d-cb08e72d65aa None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:05.233786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 851/1715] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:05.235360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c7884-0739-4fb7-85fa-ea3fdbe8ed20 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:05.240009 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd098b8f-5ffe-4c9d-834c-29d3427b341e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:05.240974 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd098b8f-5ffe-4c9d-834c-29d3427b341e admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:05.241426 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c7884-0739-4fb7-85fa-ea3fdbe8ed20 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:05.241548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd098b8f-5ffe-4c9d-834c-29d3427b341e admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:05.241957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd098b8f-5ffe-4c9d-834c-29d3427b341e admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:05.242361 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9c7884-0739-4fb7-85fa-ea3fdbe8ed20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:05.243543 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd098b8f-5ffe-4c9d-834c-29d3427b341e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:05.245868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd098b8f-5ffe-4c9d-834c-29d3427b341e admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:05.252095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:05.278480 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-dc9c7884-0739-4fb7-85fa-ea3fdbe8ed20 None None] MFA Rules not processed for user `a1b1a557a3834b98a3397c41665b5dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:05.298296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 865/1716] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:01:05 2022] POST /identity/v3/users => generated 294 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:05.299047 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd098b8f-5ffe-4c9d-834c-29d3427b341e admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:05.299267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-dc9c7884-0739-4fb7-85fa-ea3fdbe8ed20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:05.312623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd098b8f-5ffe-4c9d-834c-29d3427b341e admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:05.321814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad19b4e-43d1-42fd-94c0-ff85d6d32c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:05.325925 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad19b4e-43d1-42fd-94c0-ff85d6d32c94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:05.326662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad19b4e-43d1-42fd-94c0-ff85d6d32c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:05.327161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad19b4e-43d1-42fd-94c0-ff85d6d32c94 admin admin] PATH_INFO: `/v3/projects/2564473986f341a08bfb44f0c8a3a0ca/users/4ea33c0b3556407cb7a3f43fe5fc43d6/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:05.349803 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad19b4e-43d1-42fd-94c0-ff85d6d32c94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2564473986f341a08bfb44f0c8a3a0ca, user_id=4ea33c0b3556407cb7a3f43fe5fc43d6, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:05.352865 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad19b4e-43d1-42fd-94c0-ff85d6d32c94 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:05.376125 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 866/1717] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:05 2022] PUT /identity/v3/projects/2564473986f341a08bfb44f0c8a3a0ca/users/4ea33c0b3556407cb7a3f43fe5fc43d6/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:05.377186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad19b4e-43d1-42fd-94c0-ff85d6d32c94 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:05.400925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 852/1718] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:05.401950 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c7884-0739-4fb7-85fa-ea3fdbe8ed20 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:05.455996 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9c7884-0739-4fb7-85fa-ea3fdbe8ed20 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:05.523644 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad19b4e-43d1-42fd-94c0-ff85d6d32c94 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:05.529924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac9d35-7d21-43f6-842c-1e1545ad3198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:05.531251 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac9d35-7d21-43f6-842c-1e1545ad3198 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:05.531883 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac9d35-7d21-43f6-842c-1e1545ad3198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:05.532289 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efac9d35-7d21-43f6-842c-1e1545ad3198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:05.533244 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac9d35-7d21-43f6-842c-1e1545ad3198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:05.537938 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efac9d35-7d21-43f6-842c-1e1545ad3198 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:05.569235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 867/1719] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:05.572726 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac9d35-7d21-43f6-842c-1e1545ad3198 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:05.601022 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efac9d35-7d21-43f6-842c-1e1545ad3198 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:05.621633 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24275115-0aa7-473b-8b7b-8ad2a03e0b2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:05.622810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24275115-0aa7-473b-8b7b-8ad2a03e0b2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:05.623098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24275115-0aa7-473b-8b7b-8ad2a03e0b2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:05.623383 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24275115-0aa7-473b-8b7b-8ad2a03e0b2f admin admin] PATH_INFO: `/v3/projects/b1a49045ea794afa8d7c697052559550/users/1e320d9f3ab046718251c691efd8d36b/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:05.652243 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae43575-859b-4f31-bd32-6b65027f3508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:05.653282 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae43575-859b-4f31-bd32-6b65027f3508 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:05.653755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae43575-859b-4f31-bd32-6b65027f3508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:05.654215 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae43575-859b-4f31-bd32-6b65027f3508 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:05.655424 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24275115-0aa7-473b-8b7b-8ad2a03e0b2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1a49045ea794afa8d7c697052559550, user_id=1e320d9f3ab046718251c691efd8d36b, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:05.655841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae43575-859b-4f31-bd32-6b65027f3508 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:05.657638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae43575-859b-4f31-bd32-6b65027f3508 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:05.658715 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24275115-0aa7-473b-8b7b-8ad2a03e0b2f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:05.674859 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 853/1720] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:05.678229 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae43575-859b-4f31-bd32-6b65027f3508 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:05.684846 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 868/1721] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:05 2022] PUT /identity/v3/projects/b1a49045ea794afa8d7c697052559550/users/1e320d9f3ab046718251c691efd8d36b/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:05.693178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24275115-0aa7-473b-8b7b-8ad2a03e0b2f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:05.705284 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae43575-859b-4f31-bd32-6b65027f3508 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:05.742296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:05.782914 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24275115-0aa7-473b-8b7b-8ad2a03e0b2f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:05.790580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6cdf1-8fe4-4435-b255-50ba27310fff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:05.791484 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6cdf1-8fe4-4435-b255-50ba27310fff admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:05.792474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6cdf1-8fe4-4435-b255-50ba27310fff admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:05.792843 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c6cdf1-8fe4-4435-b255-50ba27310fff admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:05.793500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6cdf1-8fe4-4435-b255-50ba27310fff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:05.795772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c6cdf1-8fe4-4435-b255-50ba27310fff admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:05.817504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 869/1722] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:05.818479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6cdf1-8fe4-4435-b255-50ba27310fff admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:05.832512 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c6cdf1-8fe4-4435-b255-50ba27310fff admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:05.838970 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50803bf3-89ac-4ce9-ae3b-1d7121890565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:05.840454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50803bf3-89ac-4ce9-ae3b-1d7121890565 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:05.847885 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50803bf3-89ac-4ce9-ae3b-1d7121890565 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:05.849278 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50803bf3-89ac-4ce9-ae3b-1d7121890565 admin admin] PATH_INFO: `/v3/projects/b1a49045ea794afa8d7c697052559550/users/1e320d9f3ab046718251c691efd8d36b/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:05.851531 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645f4868-08d0-4cb3-aa9f-eef5ce2e8e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:05.851935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50803bf3-89ac-4ce9-ae3b-1d7121890565 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1a49045ea794afa8d7c697052559550, user_id=1e320d9f3ab046718251c691efd8d36b, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:05.852458 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f4868-08d0-4cb3-aa9f-eef5ce2e8e71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:05.852803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f4868-08d0-4cb3-aa9f-eef5ce2e8e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:05.853017 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645f4868-08d0-4cb3-aa9f-eef5ce2e8e71 admin admin] PATH_INFO: `/v3/projects/2564473986f341a08bfb44f0c8a3a0ca/users/4ea33c0b3556407cb7a3f43fe5fc43d6/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:05.855133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50803bf3-89ac-4ce9-ae3b-1d7121890565 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:05.855254 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f4868-08d0-4cb3-aa9f-eef5ce2e8e71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2564473986f341a08bfb44f0c8a3a0ca, user_id=4ea33c0b3556407cb7a3f43fe5fc43d6, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:05.857793 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645f4868-08d0-4cb3-aa9f-eef5ce2e8e71 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:05.873874 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 870/1723] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:05 2022] PUT /identity/v3/projects/b1a49045ea794afa8d7c697052559550/users/1e320d9f3ab046718251c691efd8d36b/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:05.875535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbecb9a2-747b-4498-ac13-c6c205f99e4a None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:05.875834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbecb9a2-747b-4498-ac13-c6c205f99e4a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:05.876114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbecb9a2-747b-4498-ac13-c6c205f99e4a None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:05.877464 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 871/1724] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 23:01:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 23:01:05.885662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671b1d6-6327-43f2-ac75-e30ca792d7d0 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:05.885958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671b1d6-6327-43f2-ac75-e30ca792d7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:05.886239 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e671b1d6-6327-43f2-ac75-e30ca792d7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:05.886537 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 854/1725] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:05 2022] PUT /identity/v3/projects/2564473986f341a08bfb44f0c8a3a0ca/users/4ea33c0b3556407cb7a3f43fe5fc43d6/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:05.895602 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098c0ae-e62c-4c7b-ad9c-ab30e23b6bc1 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:05.895894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098c0ae-e62c-4c7b-ad9c-ab30e23b6bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:05.896175 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1098c0ae-e62c-4c7b-ad9c-ab30e23b6bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:05.929925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-1098c0ae-e62c-4c7b-ad9c-ab30e23b6bc1 None None] MFA Rules not processed for user `4180f2593149437fa27effd8d1338d10`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:05.933236 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-1098c0ae-e62c-4c7b-ad9c-ab30e23b6bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:05.935894 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-e671b1d6-6327-43f2-ac75-e30ca792d7d0 None None] MFA Rules not processed for user `1e320d9f3ab046718251c691efd8d36b`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:05.937405 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e671b1d6-6327-43f2-ac75-e30ca792d7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:05.987258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:06.040473 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 855/1726] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 23:01:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:01:06.042169 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46515ba3-cc4f-4aef-92cc-bf490ee10f13 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:06.042912 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46515ba3-cc4f-4aef-92cc-bf490ee10f13 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:06.043404 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46515ba3-cc4f-4aef-92cc-bf490ee10f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:06.064567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 872/1727] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:05 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:06.084891 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671b1d6-6327-43f2-ac75-e30ca792d7d0 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:06.087055 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e671b1d6-6327-43f2-ac75-e30ca792d7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:06.104615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-46515ba3-cc4f-4aef-92cc-bf490ee10f13 None None] MFA Rules not processed for user `4ea33c0b3556407cb7a3f43fe5fc43d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:06.107102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-46515ba3-cc4f-4aef-92cc-bf490ee10f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:06.180242 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e671b1d6-6327-43f2-ac75-e30ca792d7d0 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:06.188445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6af7f04-1bc8-4acd-aec4-6284343b24df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:06.189255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af7f04-1bc8-4acd-aec4-6284343b24df service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:06.189644 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af7f04-1bc8-4acd-aec4-6284343b24df service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:06.190077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6af7f04-1bc8-4acd-aec4-6284343b24df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:06.262722 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:06.277933 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af7f04-1bc8-4acd-aec4-6284343b24df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:06.279545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6af7f04-1bc8-4acd-aec4-6284343b24df service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:06.297752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 856/1728] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:06 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:06.303312 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 873/1729] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:01:06.310964 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46515ba3-cc4f-4aef-92cc-bf490ee10f13 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:06.317656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-46515ba3-cc4f-4aef-92cc-bf490ee10f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:06.338218 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46515ba3-cc4f-4aef-92cc-bf490ee10f13 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:06.358774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03cc67-1dd4-4cd6-9e59-18632f86337e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:06.358774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03cc67-1dd4-4cd6-9e59-18632f86337e service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:06.359056 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03cc67-1dd4-4cd6-9e59-18632f86337e service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:06.359652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee03cc67-1dd4-4cd6-9e59-18632f86337e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:06.392133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03cc67-1dd4-4cd6-9e59-18632f86337e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:06.393663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee03cc67-1dd4-4cd6-9e59-18632f86337e service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:06.399205 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4592375-9dc9-4e77-9506-09abb3120e3c None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:06.399748 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4592375-9dc9-4e77-9506-09abb3120e3c None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:06.400251 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4592375-9dc9-4e77-9506-09abb3120e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:06.412068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 857/1730] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:06 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:06.430483 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-c4592375-9dc9-4e77-9506-09abb3120e3c None None] MFA Rules not processed for user `6b794f7aa2a7488e91f7d9ea35a45d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:06.432825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c4592375-9dc9-4e77-9506-09abb3120e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:06.532949 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03cc67-1dd4-4cd6-9e59-18632f86337e service cinder] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:06.547524 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee03cc67-1dd4-4cd6-9e59-18632f86337e service cinder] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:06.556341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80555e-ee5f-49fa-8d4b-158b70605538 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:06.556987 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80555e-ee5f-49fa-8d4b-158b70605538 service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:06.557280 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80555e-ee5f-49fa-8d4b-158b70605538 service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:06.557564 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df80555e-ee5f-49fa-8d4b-158b70605538 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:06.565154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 874/1731] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 23:01:06 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:01:06.589707 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4592375-9dc9-4e77-9506-09abb3120e3c None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:06.591521 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c4592375-9dc9-4e77-9506-09abb3120e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:06.619475 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4592375-9dc9-4e77-9506-09abb3120e3c None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:06.640700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1cab9-d14a-4ccd-95a5-28a69b6c48f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:06.641570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1cab9-d14a-4ccd-95a5-28a69b6c48f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:06.641810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1cab9-d14a-4ccd-95a5-28a69b6c48f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:06.642108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1cab9-d14a-4ccd-95a5-28a69b6c48f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:06.651238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1cab9-d14a-4ccd-95a5-28a69b6c48f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:06.652880 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef1cab9-d14a-4ccd-95a5-28a69b6c48f8 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:06.664224 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 875/1732] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:06.672219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80555e-ee5f-49fa-8d4b-158b70605538 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:06.673663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df80555e-ee5f-49fa-8d4b-158b70605538 service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:06.695458 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 858/1733] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:06 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:07.007996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1cab9-d14a-4ccd-95a5-28a69b6c48f8 service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:07.043520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1cab9-d14a-4ccd-95a5-28a69b6c48f8 service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:07.055622 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb599c2-8cdd-4063-b965-cc1846ccec2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:07.056202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb599c2-8cdd-4063-b965-cc1846ccec2b service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:07.056524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb599c2-8cdd-4063-b965-cc1846ccec2b service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:07.056808 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb599c2-8cdd-4063-b965-cc1846ccec2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:07.068150 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb599c2-8cdd-4063-b965-cc1846ccec2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:07.069779 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb599c2-8cdd-4063-b965-cc1846ccec2b service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:07.081415 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 876/1734] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:14.362042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80555e-ee5f-49fa-8d4b-158b70605538 service cinder] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:14.370463 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df80555e-ee5f-49fa-8d4b-158b70605538 service cinder] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:14.374155 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80628fb8-4633-4557-a2f4-529560306e2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:14.374836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80628fb8-4633-4557-a2f4-529560306e2f service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:14.375130 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80628fb8-4633-4557-a2f4-529560306e2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:14.375414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80628fb8-4633-4557-a2f4-529560306e2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:14.384836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80628fb8-4633-4557-a2f4-529560306e2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:14.386272 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80628fb8-4633-4557-a2f4-529560306e2f service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:14.397326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 859/1735] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:14 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:14.695190 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb599c2-8cdd-4063-b965-cc1846ccec2b service nova] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:14.779181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb599c2-8cdd-4063-b965-cc1846ccec2b service nova] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:14.785506 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c36b9-87c8-4170-8f98-1c7bd48ada17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:14.786621 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c36b9-87c8-4170-8f98-1c7bd48ada17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:14.786911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c36b9-87c8-4170-8f98-1c7bd48ada17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:14.787193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c36b9-87c8-4170-8f98-1c7bd48ada17 admin admin] PATH_INFO: `/v3/users/5b21f9e2b5434b00b24f0aeb046f6e50` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:14.788420 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c36b9-87c8-4170-8f98-1c7bd48ada17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b21f9e2b5434b00b24f0aeb046f6e50)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:14.790367 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c36b9-87c8-4170-8f98-1c7bd48ada17 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:14.867109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-b79c36b9-87c8-4170-8f98-1c7bd48ada17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b21f9e2b5434b00b24f0aeb046f6e50'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:14.884202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-b79c36b9-87c8-4170-8f98-1c7bd48ada17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b21f9e2b5434b00b24f0aeb046f6e50'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:14.884989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-b79c36b9-87c8-4170-8f98-1c7bd48ada17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b21f9e2b5434b00b24f0aeb046f6e50'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:14.902536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-b79c36b9-87c8-4170-8f98-1c7bd48ada17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b21f9e2b5434b00b24f0aeb046f6e50'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:14.903345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-b79c36b9-87c8-4170-8f98-1c7bd48ada17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b21f9e2b5434b00b24f0aeb046f6e50'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:14.908433 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:14.919577 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 877/1736] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:01:14 2022] DELETE /identity/v3/users/5b21f9e2b5434b00b24f0aeb046f6e50 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:01:14.924285 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da51ba-c291-47c9-b4d2-9944716ad093 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:14.925267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da51ba-c291-47c9-b4d2-9944716ad093 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:14.925625 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da51ba-c291-47c9-b4d2-9944716ad093 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:14.944120 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-c0da51ba-c291-47c9-b4d2-9944716ad093 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:14.945541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c0da51ba-c291-47c9-b4d2-9944716ad093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:14.972378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:15.012233 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 860/1737] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:15.022297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c36b9-87c8-4170-8f98-1c7bd48ada17 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:15.023590 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b79c36b9-87c8-4170-8f98-1c7bd48ada17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:15.083831 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c36b9-87c8-4170-8f98-1c7bd48ada17 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:15.087459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf8136-3767-47fa-9adf-a6fc6293934d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:15.088117 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf8136-3767-47fa-9adf-a6fc6293934d service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:15.088407 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf8136-3767-47fa-9adf-a6fc6293934d service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:15.088691 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf8136-3767-47fa-9adf-a6fc6293934d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:15.098151 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf8136-3767-47fa-9adf-a6fc6293934d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:15.099640 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf8136-3767-47fa-9adf-a6fc6293934d service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:15.109212 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 878/1738] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:15.377927 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da51ba-c291-47c9-b4d2-9944716ad093 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:15.379429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c0da51ba-c291-47c9-b4d2-9944716ad093 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:15.392875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0da51ba-c291-47c9-b4d2-9944716ad093 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:15.398776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296abb7-a82d-4f80-a0f3-07666df379e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:15.399664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296abb7-a82d-4f80-a0f3-07666df379e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:15.399947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296abb7-a82d-4f80-a0f3-07666df379e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:15.400224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296abb7-a82d-4f80-a0f3-07666df379e4 admin admin] PATH_INFO: `/v3/projects/dec4af79ecf54d3bad6c9aea58ba159f` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:15.401435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296abb7-a82d-4f80-a0f3-07666df379e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dec4af79ecf54d3bad6c9aea58ba159f)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:15.403429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296abb7-a82d-4f80-a0f3-07666df379e4 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:15.455967 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c296abb7-a82d-4f80-a0f3-07666df379e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dec4af79ecf54d3bad6c9aea58ba159f', 'request_id': 'req-c296abb7-a82d-4f80-a0f3-07666df379e4'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:15.463164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c296abb7-a82d-4f80-a0f3-07666df379e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dec4af79ecf54d3bad6c9aea58ba159f', 'request_id': 'req-c296abb7-a82d-4f80-a0f3-07666df379e4'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:15.470333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c296abb7-a82d-4f80-a0f3-07666df379e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dec4af79ecf54d3bad6c9aea58ba159f', 'request_id': 'req-c296abb7-a82d-4f80-a0f3-07666df379e4'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:15.486366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c296abb7-a82d-4f80-a0f3-07666df379e4 admin admin] The token cache is being invalidate because project dec4af79ecf54d3bad6c9aea58ba159f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:01:15.487492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c296abb7-a82d-4f80-a0f3-07666df379e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:15.488220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c296abb7-a82d-4f80-a0f3-07666df379e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:15.492624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 861/1739] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:01:15 2022] DELETE /identity/v3/projects/dec4af79ecf54d3bad6c9aea58ba159f => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:15.494304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf8136-3767-47fa-9adf-a6fc6293934d service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:15.504956 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf8136-3767-47fa-9adf-a6fc6293934d service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:15.570390 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b53a64a-a827-456c-8b1f-c7871e28c9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:15.571238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53a64a-a827-456c-8b1f-c7871e28c9d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:15.571632 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53a64a-a827-456c-8b1f-c7871e28c9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:15.571632 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b53a64a-a827-456c-8b1f-c7871e28c9d0 admin admin] PATH_INFO: `/v3/users/4ea33c0b3556407cb7a3f43fe5fc43d6` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:15.572899 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b53a64a-a827-456c-8b1f-c7871e28c9d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ea33c0b3556407cb7a3f43fe5fc43d6)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:15.574868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b53a64a-a827-456c-8b1f-c7871e28c9d0 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:15.633750 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8b53a64a-a827-456c-8b1f-c7871e28c9d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ea33c0b3556407cb7a3f43fe5fc43d6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:15.638877 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8b53a64a-a827-456c-8b1f-c7871e28c9d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ea33c0b3556407cb7a3f43fe5fc43d6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:15.639658 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8b53a64a-a827-456c-8b1f-c7871e28c9d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ea33c0b3556407cb7a3f43fe5fc43d6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:15.654624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8b53a64a-a827-456c-8b1f-c7871e28c9d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ea33c0b3556407cb7a3f43fe5fc43d6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:15.655371 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8b53a64a-a827-456c-8b1f-c7871e28c9d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ea33c0b3556407cb7a3f43fe5fc43d6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:15.670348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 879/1740] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:01:15 2022] DELETE /identity/v3/users/4ea33c0b3556407cb7a3f43fe5fc43d6 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:15.958652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296abb7-a82d-4f80-a0f3-07666df379e4 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:15.960130 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c296abb7-a82d-4f80-a0f3-07666df379e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:15.973937 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296abb7-a82d-4f80-a0f3-07666df379e4 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:16.049074 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310e23fc-5f56-4ef0-a776-f4cd27cc5387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:16.050811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e23fc-5f56-4ef0-a776-f4cd27cc5387 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:16.051103 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e23fc-5f56-4ef0-a776-f4cd27cc5387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:16.051382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310e23fc-5f56-4ef0-a776-f4cd27cc5387 admin admin] PATH_INFO: `/v3/projects/2564473986f341a08bfb44f0c8a3a0ca` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:16.052611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e23fc-5f56-4ef0-a776-f4cd27cc5387 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2564473986f341a08bfb44f0c8a3a0ca)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:16.054647 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310e23fc-5f56-4ef0-a776-f4cd27cc5387 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:16.113839 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-310e23fc-5f56-4ef0-a776-f4cd27cc5387 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2564473986f341a08bfb44f0c8a3a0ca', 'request_id': 'req-310e23fc-5f56-4ef0-a776-f4cd27cc5387'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:16.120997 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-310e23fc-5f56-4ef0-a776-f4cd27cc5387 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2564473986f341a08bfb44f0c8a3a0ca', 'request_id': 'req-310e23fc-5f56-4ef0-a776-f4cd27cc5387'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:16.122784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:16.131613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-310e23fc-5f56-4ef0-a776-f4cd27cc5387 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2564473986f341a08bfb44f0c8a3a0ca', 'request_id': 'req-310e23fc-5f56-4ef0-a776-f4cd27cc5387'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:16.146086 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-310e23fc-5f56-4ef0-a776-f4cd27cc5387 admin admin] The token cache is being invalidate because project 2564473986f341a08bfb44f0c8a3a0ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:01:16.146376 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-310e23fc-5f56-4ef0-a776-f4cd27cc5387 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:16.147031 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-310e23fc-5f56-4ef0-a776-f4cd27cc5387 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:16.148579 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 862/1741] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:01:15 2022] DELETE /identity/v3/projects/2564473986f341a08bfb44f0c8a3a0ca => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:16.153630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422148c-44e0-4c0d-b3a8-235a53513c0b None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:16.154083 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422148c-44e0-4c0d-b3a8-235a53513c0b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:16.154213 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422148c-44e0-4c0d-b3a8-235a53513c0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:16.173099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-1422148c-44e0-4c0d-b3a8-235a53513c0b None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:16.174352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-1422148c-44e0-4c0d-b3a8-235a53513c0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:16.187358 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:16.240815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 880/1742] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:16.248707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ef2ed-bdf0-4363-8b94-fe7a1f7d2055 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:16.249042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ef2ed-bdf0-4363-8b94-fe7a1f7d2055 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:16.249317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377ef2ed-bdf0-4363-8b94-fe7a1f7d2055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:16.267416 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-377ef2ed-bdf0-4363-8b94-fe7a1f7d2055 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:16.268647 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-377ef2ed-bdf0-4363-8b94-fe7a1f7d2055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:16.276677 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 863/1743] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:16.281115 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422148c-44e0-4c0d-b3a8-235a53513c0b None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:16.291367 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422148c-44e0-4c0d-b3a8-235a53513c0b None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:16.295940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98148eec-41f0-4da1-8278-d7e8941a8dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:16.296592 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98148eec-41f0-4da1-8278-d7e8941a8dbc admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:16.296876 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98148eec-41f0-4da1-8278-d7e8941a8dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:16.297149 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98148eec-41f0-4da1-8278-d7e8941a8dbc admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:16.297867 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98148eec-41f0-4da1-8278-d7e8941a8dbc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:01:16.298173 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98148eec-41f0-4da1-8278-d7e8941a8dbc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:16.300414 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98148eec-41f0-4da1-8278-d7e8941a8dbc admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:16.311641 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 881/1744] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 23:01:16.315669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ef2ed-bdf0-4363-8b94-fe7a1f7d2055 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:16.325960 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377ef2ed-bdf0-4363-8b94-fe7a1f7d2055 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:16.330511 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d2bd7-e8d5-484f-a45a-1d4390f783ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:16.331192 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d2bd7-e8d5-484f-a45a-1d4390f783ac admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:16.331476 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d2bd7-e8d5-484f-a45a-1d4390f783ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:16.331756 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31d2bd7-e8d5-484f-a45a-1d4390f783ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:16.332440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d2bd7-e8d5-484f-a45a-1d4390f783ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:16.334308 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31d2bd7-e8d5-484f-a45a-1d4390f783ac admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:16.360658 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 864/1745] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:01:16 2022] POST /identity/v3/projects => generated 383 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:01:16.364867 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98148eec-41f0-4da1-8278-d7e8941a8dbc admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:16.374771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98148eec-41f0-4da1-8278-d7e8941a8dbc admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:16.431236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52ad4d-dfd1-4af5-b411-dd46c79e895d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:16.431864 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52ad4d-dfd1-4af5-b411-dd46c79e895d admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:16.432175 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52ad4d-dfd1-4af5-b411-dd46c79e895d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:16.432476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a52ad4d-dfd1-4af5-b411-dd46c79e895d admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:16.433163 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52ad4d-dfd1-4af5-b411-dd46c79e895d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:16.435270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a52ad4d-dfd1-4af5-b411-dd46c79e895d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:16.457195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 882/1746] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:01:16 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:01:16.461199 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d2bd7-e8d5-484f-a45a-1d4390f783ac admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:16.472217 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31d2bd7-e8d5-484f-a45a-1d4390f783ac admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:16.476726 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ed5d7-6060-4da5-87bf-dee152317feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:16.477389 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ed5d7-6060-4da5-87bf-dee152317feb admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:16.477678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ed5d7-6060-4da5-87bf-dee152317feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:16.477963 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01ed5d7-6060-4da5-87bf-dee152317feb admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:16.478722 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ed5d7-6060-4da5-87bf-dee152317feb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:16.480315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01ed5d7-6060-4da5-87bf-dee152317feb admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:16.491419 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 865/1747] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:16.495164 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52ad4d-dfd1-4af5-b411-dd46c79e895d admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:16.505389 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a52ad4d-dfd1-4af5-b411-dd46c79e895d admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:16.509917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19d1355-f410-4b09-9af4-8ef78f2d7026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:16.510573 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d1355-f410-4b09-9af4-8ef78f2d7026 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:16.510887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d1355-f410-4b09-9af4-8ef78f2d7026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:16.511164 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19d1355-f410-4b09-9af4-8ef78f2d7026 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:16.511883 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d1355-f410-4b09-9af4-8ef78f2d7026 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:16.513440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19d1355-f410-4b09-9af4-8ef78f2d7026 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:16.523155 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 883/1748] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:16.527237 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ed5d7-6060-4da5-87bf-dee152317feb admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:16.537490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01ed5d7-6060-4da5-87bf-dee152317feb admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:16.542247 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f19f74-a4cf-46ce-8c9c-bf31dc39add8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:16.543243 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f19f74-a4cf-46ce-8c9c-bf31dc39add8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:16.543535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f19f74-a4cf-46ce-8c9c-bf31dc39add8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:16.543868 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f19f74-a4cf-46ce-8c9c-bf31dc39add8 admin admin] PATH_INFO: `/v3/projects/fba54b379b81417fa5c1c21b31f0cd05/users/3aab9dee3a0c429c97fa67f504504c33/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:16.558956 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f19f74-a4cf-46ce-8c9c-bf31dc39add8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fba54b379b81417fa5c1c21b31f0cd05, user_id=3aab9dee3a0c429c97fa67f504504c33, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:16.561600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f19f74-a4cf-46ce-8c9c-bf31dc39add8 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:16.575627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 866/1749] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:16 2022] PUT /identity/v3/projects/fba54b379b81417fa5c1c21b31f0cd05/users/3aab9dee3a0c429c97fa67f504504c33/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:16.587280 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d30c17-fc92-49ba-b618-3c61f8785627 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:16.587645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d30c17-fc92-49ba-b618-3c61f8785627 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:16.588004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d30c17-fc92-49ba-b618-3c61f8785627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:16.615547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a0d30c17-fc92-49ba-b618-3c61f8785627 None None] MFA Rules not processed for user `3aab9dee3a0c429c97fa67f504504c33`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:16.616887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a0d30c17-fc92-49ba-b618-3c61f8785627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:16.687509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 884/1750] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:16.692605 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f19f74-a4cf-46ce-8c9c-bf31dc39add8 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:16.761619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f19f74-a4cf-46ce-8c9c-bf31dc39add8 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:16.766237 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae307da3-113d-4ed8-9761-5203f5a09d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:16.766958 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae307da3-113d-4ed8-9761-5203f5a09d9a admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:16.767252 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae307da3-113d-4ed8-9761-5203f5a09d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:16.767536 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae307da3-113d-4ed8-9761-5203f5a09d9a admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:16.768308 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae307da3-113d-4ed8-9761-5203f5a09d9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:16.770285 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae307da3-113d-4ed8-9761-5203f5a09d9a admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:16.799453 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 867/1751] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:01:16 2022] POST /identity/v3/projects => generated 381 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:16.800616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d30c17-fc92-49ba-b618-3c61f8785627 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:16.812363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d30c17-fc92-49ba-b618-3c61f8785627 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:16.828387 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:16.880814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08a879-bf4f-4416-a074-3a8f33cf4758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:16.881549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08a879-bf4f-4416-a074-3a8f33cf4758 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:16.881841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08a879-bf4f-4416-a074-3a8f33cf4758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:16.882213 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08a879-bf4f-4416-a074-3a8f33cf4758 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:16.882884 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08a879-bf4f-4416-a074-3a8f33cf4758 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:16.884958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08a879-bf4f-4416-a074-3a8f33cf4758 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:16.911300 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 885/1752] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:01:16 2022] POST /identity/v3/users => generated 297 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:01:16.915045 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae307da3-113d-4ed8-9761-5203f5a09d9a admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:16.925897 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae307da3-113d-4ed8-9761-5203f5a09d9a admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:16.930447 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5b6a0-590c-4178-9e93-2cc2e7bd63cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:16.931152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5b6a0-590c-4178-9e93-2cc2e7bd63cc admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:16.931449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5b6a0-590c-4178-9e93-2cc2e7bd63cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:16.931733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5b6a0-590c-4178-9e93-2cc2e7bd63cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:16.932462 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5b6a0-590c-4178-9e93-2cc2e7bd63cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:16.934081 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5b6a0-590c-4178-9e93-2cc2e7bd63cc admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:16.946050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 868/1753] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:16.950171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08a879-bf4f-4416-a074-3a8f33cf4758 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:16.960545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08a879-bf4f-4416-a074-3a8f33cf4758 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:16.965433 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c9177-6183-42e7-9f27-12251e656aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:16.966358 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c9177-6183-42e7-9f27-12251e656aa3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:16.971491 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c9177-6183-42e7-9f27-12251e656aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:16.971797 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c9177-6183-42e7-9f27-12251e656aa3 admin admin] PATH_INFO: `/v3/projects/aabb836552bb40c884606be02cf0e0c9/users/728d5b52631441569b00a2055754d755/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:16.988201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c9177-6183-42e7-9f27-12251e656aa3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aabb836552bb40c884606be02cf0e0c9, user_id=728d5b52631441569b00a2055754d755, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:16.990987 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c9177-6183-42e7-9f27-12251e656aa3 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:17.005884 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 886/1754] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:16 2022] PUT /identity/v3/projects/aabb836552bb40c884606be02cf0e0c9/users/728d5b52631441569b00a2055754d755/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:17.009778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5b6a0-590c-4178-9e93-2cc2e7bd63cc admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:17.020289 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5b6a0-590c-4178-9e93-2cc2e7bd63cc admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:17.074248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:17.077438 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126a7d1c-5a15-4ff0-a6a4-c86e0333fb83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:17.078293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a7d1c-5a15-4ff0-a6a4-c86e0333fb83 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:17.078797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a7d1c-5a15-4ff0-a6a4-c86e0333fb83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:17.079250 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126a7d1c-5a15-4ff0-a6a4-c86e0333fb83 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:17.080053 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a7d1c-5a15-4ff0-a6a4-c86e0333fb83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:17.082254 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126a7d1c-5a15-4ff0-a6a4-c86e0333fb83 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:17.093802 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 869/1755] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:17.098425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c9177-6183-42e7-9f27-12251e656aa3 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:17.108862 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c9177-6183-42e7-9f27-12251e656aa3 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:17.114229 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32bec9-11f0-48b1-bef2-81e7e6b9e3f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:17.115211 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32bec9-11f0-48b1-bef2-81e7e6b9e3f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:17.115514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32bec9-11f0-48b1-bef2-81e7e6b9e3f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:17.115797 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f32bec9-11f0-48b1-bef2-81e7e6b9e3f3 admin admin] PATH_INFO: `/v3/projects/aabb836552bb40c884606be02cf0e0c9/users/728d5b52631441569b00a2055754d755/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:17.117976 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32bec9-11f0-48b1-bef2-81e7e6b9e3f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aabb836552bb40c884606be02cf0e0c9, user_id=728d5b52631441569b00a2055754d755, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:17.120752 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f32bec9-11f0-48b1-bef2-81e7e6b9e3f3 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:17.131902 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 887/1756] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:17 2022] PUT /identity/v3/projects/aabb836552bb40c884606be02cf0e0c9/users/728d5b52631441569b00a2055754d755/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:17.141597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681df60-5af8-4fd8-ae1a-fa8255a8f09f None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:17.141893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681df60-5af8-4fd8-ae1a-fa8255a8f09f None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:17.142178 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b681df60-5af8-4fd8-ae1a-fa8255a8f09f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:17.178631 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-b681df60-5af8-4fd8-ae1a-fa8255a8f09f None None] MFA Rules not processed for user `728d5b52631441569b00a2055754d755`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:17.180707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-b681df60-5af8-4fd8-ae1a-fa8255a8f09f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:17.258808 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 870/1757] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:17.269964 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32bec9-11f0-48b1-bef2-81e7e6b9e3f3 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:17.272431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7f32bec9-11f0-48b1-bef2-81e7e6b9e3f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:17.329803 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f32bec9-11f0-48b1-bef2-81e7e6b9e3f3 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:17.338959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95b2b79-1826-429a-a84e-4bddba37f650 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:17.339988 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b2b79-1826-429a-a84e-4bddba37f650 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:17.340273 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b2b79-1826-429a-a84e-4bddba37f650 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:17.340548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95b2b79-1826-429a-a84e-4bddba37f650 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:17.354811 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b2b79-1826-429a-a84e-4bddba37f650 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:17.356214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95b2b79-1826-429a-a84e-4bddba37f650 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:17.366336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 888/1758] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:17.469634 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681df60-5af8-4fd8-ae1a-fa8255a8f09f None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:17.478877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b681df60-5af8-4fd8-ae1a-fa8255a8f09f None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:17.482632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239b29c7-874d-40f0-a1b0-f048169e049a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:17.483276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b29c7-874d-40f0-a1b0-f048169e049a service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:17.483599 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b29c7-874d-40f0-a1b0-f048169e049a service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:17.483838 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239b29c7-874d-40f0-a1b0-f048169e049a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:17.541406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b29c7-874d-40f0-a1b0-f048169e049a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:17.542877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239b29c7-874d-40f0-a1b0-f048169e049a service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:17.552054 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 871/1759] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:19.249453 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c45a1f-5935-477b-933d-57a6af123369 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:19.249965 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c45a1f-5935-477b-933d-57a6af123369 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:19.250203 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c45a1f-5935-477b-933d-57a6af123369 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:19.252627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 889/1760] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 23:01:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 23:01:19.328660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ef0a5-1a83-45dd-9ef8-9d42d7ea5afd None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:19.329205 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ef0a5-1a83-45dd-9ef8-9d42d7ea5afd None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:19.329990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15ef0a5-1a83-45dd-9ef8-9d42d7ea5afd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:19.428623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-c15ef0a5-1a83-45dd-9ef8-9d42d7ea5afd None None] MFA Rules not processed for user `4180f2593149437fa27effd8d1338d10`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:19.435626 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c15ef0a5-1a83-45dd-9ef8-9d42d7ea5afd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:19.678224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 872/1761] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 23:01:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:20.015002 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c45a1f-5935-477b-933d-57a6af123369 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:20.019204 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e4c45a1f-5935-477b-933d-57a6af123369 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:20.042576 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c45a1f-5935-477b-933d-57a6af123369 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:20.052726 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b743aad7-d706-499e-bdbc-8b39c5dea6ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:20.053851 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743aad7-d706-499e-bdbc-8b39c5dea6ae service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:20.054449 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743aad7-d706-499e-bdbc-8b39c5dea6ae service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:20.055124 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b743aad7-d706-499e-bdbc-8b39c5dea6ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:20.072603 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b743aad7-d706-499e-bdbc-8b39c5dea6ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:20.074889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b743aad7-d706-499e-bdbc-8b39c5dea6ae service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:20.090811 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 890/1762] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:20.307208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ef0a5-1a83-45dd-9ef8-9d42d7ea5afd None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:20.313437 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:20.382818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15ef0a5-1a83-45dd-9ef8-9d42d7ea5afd None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:20.388582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91800a8d-e674-4a55-9f40-80678b752576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:20.389569 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91800a8d-e674-4a55-9f40-80678b752576 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:20.390384 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91800a8d-e674-4a55-9f40-80678b752576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:20.390632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91800a8d-e674-4a55-9f40-80678b752576 admin admin] PATH_INFO: `/v3/users/3aab9dee3a0c429c97fa67f504504c33` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:20.392136 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91800a8d-e674-4a55-9f40-80678b752576 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aab9dee3a0c429c97fa67f504504c33)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:20.394978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91800a8d-e674-4a55-9f40-80678b752576 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:20.465267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-91800a8d-e674-4a55-9f40-80678b752576 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aab9dee3a0c429c97fa67f504504c33'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:20.465932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-91800a8d-e674-4a55-9f40-80678b752576 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aab9dee3a0c429c97fa67f504504c33'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:20.473438 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-91800a8d-e674-4a55-9f40-80678b752576 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aab9dee3a0c429c97fa67f504504c33'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:20.474115 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-91800a8d-e674-4a55-9f40-80678b752576 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aab9dee3a0c429c97fa67f504504c33'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:20.488603 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-91800a8d-e674-4a55-9f40-80678b752576 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aab9dee3a0c429c97fa67f504504c33'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:20.503009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 873/1763] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:01:20 2022] DELETE /identity/v3/users/3aab9dee3a0c429c97fa67f504504c33 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:20.507198 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7c0ea-f8bf-43e4-a270-587e75584522 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:20.507789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7c0ea-f8bf-43e4-a270-587e75584522 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:20.507888 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7c0ea-f8bf-43e4-a270-587e75584522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:20.530732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-f3a7c0ea-f8bf-43e4-a270-587e75584522 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:20.533697 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-f3a7c0ea-f8bf-43e4-a270-587e75584522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:20.586666 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:20.600954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 891/1764] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:20.611049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91800a8d-e674-4a55-9f40-80678b752576 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:20.613191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-91800a8d-e674-4a55-9f40-80678b752576 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:20.671411 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91800a8d-e674-4a55-9f40-80678b752576 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:20.675762 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be2f33-2b81-4c5c-b48f-5b9063a31539 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:20.676422 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be2f33-2b81-4c5c-b48f-5b9063a31539 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:20.676711 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be2f33-2b81-4c5c-b48f-5b9063a31539 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:20.676995 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38be2f33-2b81-4c5c-b48f-5b9063a31539 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:20.686309 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be2f33-2b81-4c5c-b48f-5b9063a31539 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:20.687808 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38be2f33-2b81-4c5c-b48f-5b9063a31539 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:20.697842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 874/1765] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:20.981463 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7c0ea-f8bf-43e4-a270-587e75584522 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:20.984078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-f3a7c0ea-f8bf-43e4-a270-587e75584522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:20.997793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7c0ea-f8bf-43e4-a270-587e75584522 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:21.004338 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90f39e-0558-44ef-9276-0e90ee0a2ba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:21.005369 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90f39e-0558-44ef-9276-0e90ee0a2ba6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:21.005679 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90f39e-0558-44ef-9276-0e90ee0a2ba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:21.005947 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90f39e-0558-44ef-9276-0e90ee0a2ba6 admin admin] PATH_INFO: `/v3/projects/fba54b379b81417fa5c1c21b31f0cd05` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:21.007178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90f39e-0558-44ef-9276-0e90ee0a2ba6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fba54b379b81417fa5c1c21b31f0cd05)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:21.009135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90f39e-0558-44ef-9276-0e90ee0a2ba6 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:21.066880 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-aa90f39e-0558-44ef-9276-0e90ee0a2ba6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fba54b379b81417fa5c1c21b31f0cd05', 'request_id': 'req-aa90f39e-0558-44ef-9276-0e90ee0a2ba6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:21.074185 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-aa90f39e-0558-44ef-9276-0e90ee0a2ba6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fba54b379b81417fa5c1c21b31f0cd05', 'request_id': 'req-aa90f39e-0558-44ef-9276-0e90ee0a2ba6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:21.081631 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-aa90f39e-0558-44ef-9276-0e90ee0a2ba6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fba54b379b81417fa5c1c21b31f0cd05', 'request_id': 'req-aa90f39e-0558-44ef-9276-0e90ee0a2ba6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:21.094258 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-aa90f39e-0558-44ef-9276-0e90ee0a2ba6 admin admin] The token cache is being invalidate because project fba54b379b81417fa5c1c21b31f0cd05 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:01:21.094636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-aa90f39e-0558-44ef-9276-0e90ee0a2ba6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:21.095211 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-aa90f39e-0558-44ef-9276-0e90ee0a2ba6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:21.096805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 892/1766] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:01:20 2022] DELETE /identity/v3/projects/fba54b379b81417fa5c1c21b31f0cd05 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:21.100867 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be2f33-2b81-4c5c-b48f-5b9063a31539 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:21.110867 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38be2f33-2b81-4c5c-b48f-5b9063a31539 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:21.140420 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:21.184175 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907c43c-4347-4f17-b641-4216be2ad70d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:21.185414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907c43c-4347-4f17-b641-4216be2ad70d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:21.185773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907c43c-4347-4f17-b641-4216be2ad70d admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:21.186361 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907c43c-4347-4f17-b641-4216be2ad70d admin admin] PATH_INFO: `/v3/users/728d5b52631441569b00a2055754d755` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:21.188006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907c43c-4347-4f17-b641-4216be2ad70d admin admin] RBAC: Authorizing `identity:delete_user(user_id=728d5b52631441569b00a2055754d755)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:21.190117 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4907c43c-4347-4f17-b641-4216be2ad70d admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:21.280505 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-4907c43c-4347-4f17-b641-4216be2ad70d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '728d5b52631441569b00a2055754d755'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:21.283171 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-4907c43c-4347-4f17-b641-4216be2ad70d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '728d5b52631441569b00a2055754d755'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:21.295976 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-4907c43c-4347-4f17-b641-4216be2ad70d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '728d5b52631441569b00a2055754d755'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:21.296772 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-4907c43c-4347-4f17-b641-4216be2ad70d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '728d5b52631441569b00a2055754d755'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:21.311474 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-4907c43c-4347-4f17-b641-4216be2ad70d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '728d5b52631441569b00a2055754d755'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:21.325401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 875/1767] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:01:21 2022] DELETE /identity/v3/users/728d5b52631441569b00a2055754d755 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:21.600198 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90f39e-0558-44ef-9276-0e90ee0a2ba6 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:21.601973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-aa90f39e-0558-44ef-9276-0e90ee0a2ba6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:21.616620 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90f39e-0558-44ef-9276-0e90ee0a2ba6 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:21.696555 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62293664-62a3-4d77-8420-f3ddcbb606b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:21.697600 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62293664-62a3-4d77-8420-f3ddcbb606b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:21.697895 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62293664-62a3-4d77-8420-f3ddcbb606b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:21.698178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62293664-62a3-4d77-8420-f3ddcbb606b9 admin admin] PATH_INFO: `/v3/projects/aabb836552bb40c884606be02cf0e0c9` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:21.699461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62293664-62a3-4d77-8420-f3ddcbb606b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aabb836552bb40c884606be02cf0e0c9)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:21.701527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62293664-62a3-4d77-8420-f3ddcbb606b9 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:21.761745 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-62293664-62a3-4d77-8420-f3ddcbb606b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aabb836552bb40c884606be02cf0e0c9', 'request_id': 'req-62293664-62a3-4d77-8420-f3ddcbb606b9'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:21.770480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-62293664-62a3-4d77-8420-f3ddcbb606b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aabb836552bb40c884606be02cf0e0c9', 'request_id': 'req-62293664-62a3-4d77-8420-f3ddcbb606b9'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:21.777310 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-62293664-62a3-4d77-8420-f3ddcbb606b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aabb836552bb40c884606be02cf0e0c9', 'request_id': 'req-62293664-62a3-4d77-8420-f3ddcbb606b9'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:21.792542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-62293664-62a3-4d77-8420-f3ddcbb606b9 admin admin] The token cache is being invalidate because project aabb836552bb40c884606be02cf0e0c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:01:21.792782 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-62293664-62a3-4d77-8420-f3ddcbb606b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:21.793728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-62293664-62a3-4d77-8420-f3ddcbb606b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:21.796661 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 893/1768] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:01:21 2022] DELETE /identity/v3/projects/aabb836552bb40c884606be02cf0e0c9 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:21.801803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cedd1b-d1bd-4859-9125-22a3e47dc363 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:21.802273 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cedd1b-d1bd-4859-9125-22a3e47dc363 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:21.802355 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cedd1b-d1bd-4859-9125-22a3e47dc363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:21.822089 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-d3cedd1b-d1bd-4859-9125-22a3e47dc363 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:21.823372 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d3cedd1b-d1bd-4859-9125-22a3e47dc363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:21.888688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 876/1769] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:01:21.896328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36faacc-85ce-4996-a6a7-90b4790a6482 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:21.896630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36faacc-85ce-4996-a6a7-90b4790a6482 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:21.896915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36faacc-85ce-4996-a6a7-90b4790a6482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:21.915572 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a36faacc-85ce-4996-a6a7-90b4790a6482 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:21.916753 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a36faacc-85ce-4996-a6a7-90b4790a6482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:21.925015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 894/1770] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:21.928951 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cedd1b-d1bd-4859-9125-22a3e47dc363 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:21.938502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cedd1b-d1bd-4859-9125-22a3e47dc363 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:21.943664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231d830-7fa2-479f-be92-5d2884c09c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:21.944320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231d830-7fa2-479f-be92-5d2884c09c3e admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:21.944607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231d830-7fa2-479f-be92-5d2884c09c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:21.944891 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231d830-7fa2-479f-be92-5d2884c09c3e admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:21.945615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231d830-7fa2-479f-be92-5d2884c09c3e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:01:21.945930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231d830-7fa2-479f-be92-5d2884c09c3e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:21.947575 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231d830-7fa2-479f-be92-5d2884c09c3e admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:21.959280 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 877/1771] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:01:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:01:21.963201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36faacc-85ce-4996-a6a7-90b4790a6482 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:21.967521 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:21.973274 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36faacc-85ce-4996-a6a7-90b4790a6482 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:21.978792 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b59b0-c090-4758-a183-7d08e2a0d589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:21.979655 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b59b0-c090-4758-a183-7d08e2a0d589 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:21.980133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b59b0-c090-4758-a183-7d08e2a0d589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:21.980477 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b59b0-c090-4758-a183-7d08e2a0d589 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:21.981414 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b59b0-c090-4758-a183-7d08e2a0d589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:21.983712 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b59b0-c090-4758-a183-7d08e2a0d589 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:22.009526 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 895/1772] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:01:21 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:22.013715 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231d830-7fa2-479f-be92-5d2884c09c3e admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:22.023673 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231d830-7fa2-479f-be92-5d2884c09c3e admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:22.055044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:22.083531 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a5767-a48e-4b6b-8c38-e2952adbdf3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:22.084212 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a5767-a48e-4b6b-8c38-e2952adbdf3b admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:22.084498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a5767-a48e-4b6b-8c38-e2952adbdf3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:22.084775 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7a5767-a48e-4b6b-8c38-e2952adbdf3b admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:22.085456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a5767-a48e-4b6b-8c38-e2952adbdf3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:22.087398 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7a5767-a48e-4b6b-8c38-e2952adbdf3b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:22.106866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 878/1773] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:01:22 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:22.110909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b59b0-c090-4758-a183-7d08e2a0d589 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:22.121589 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b59b0-c090-4758-a183-7d08e2a0d589 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:22.126162 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f8033-56fd-4592-9dd7-61eeed6444ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:22.126918 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f8033-56fd-4592-9dd7-61eeed6444ff admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:22.127208 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f8033-56fd-4592-9dd7-61eeed6444ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:22.127493 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7f8033-56fd-4592-9dd7-61eeed6444ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:22.128244 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f8033-56fd-4592-9dd7-61eeed6444ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:22.129832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7f8033-56fd-4592-9dd7-61eeed6444ff admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:22.140198 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 896/1774] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:22.144064 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a5767-a48e-4b6b-8c38-e2952adbdf3b admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:22.153971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7a5767-a48e-4b6b-8c38-e2952adbdf3b admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:22.159034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa32fc-3d84-403d-b85d-7867d83db324 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:22.159689 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa32fc-3d84-403d-b85d-7867d83db324 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:22.159977 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa32fc-3d84-403d-b85d-7867d83db324 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:22.160255 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa32fc-3d84-403d-b85d-7867d83db324 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:22.160970 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa32fc-3d84-403d-b85d-7867d83db324 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:22.162537 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa32fc-3d84-403d-b85d-7867d83db324 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:22.172388 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 879/1775] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:22.176847 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f8033-56fd-4592-9dd7-61eeed6444ff admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:22.186779 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7f8033-56fd-4592-9dd7-61eeed6444ff admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:22.191961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9df75-727b-4ad4-9f3b-be10a832aacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:22.192846 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9df75-727b-4ad4-9f3b-be10a832aacb admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:22.193136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9df75-727b-4ad4-9f3b-be10a832aacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:22.193414 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd9df75-727b-4ad4-9f3b-be10a832aacb admin admin] PATH_INFO: `/v3/projects/4246f6446ef14446bf754b904f2e685a/users/f53ab2860eea4264b469aea3da49e637/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:22.209074 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd9df75-727b-4ad4-9f3b-be10a832aacb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4246f6446ef14446bf754b904f2e685a, user_id=f53ab2860eea4264b469aea3da49e637, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:22.211886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd9df75-727b-4ad4-9f3b-be10a832aacb admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:22.226672 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 897/1776] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:22 2022] PUT /identity/v3/projects/4246f6446ef14446bf754b904f2e685a/users/f53ab2860eea4264b469aea3da49e637/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:22.238374 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bee012-c576-4342-a74b-e26ab65b59f8 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:22.238709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bee012-c576-4342-a74b-e26ab65b59f8 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:22.238995 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bee012-c576-4342-a74b-e26ab65b59f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:22.265542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-60bee012-c576-4342-a74b-e26ab65b59f8 None None] MFA Rules not processed for user `f53ab2860eea4264b469aea3da49e637`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:22.266820 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-60bee012-c576-4342-a74b-e26ab65b59f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:22.335658 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 880/1777] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:22.339879 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9df75-727b-4ad4-9f3b-be10a832aacb admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:22.405629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd9df75-727b-4ad4-9f3b-be10a832aacb admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:22.410911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c24de-1e9c-408e-8dcb-6c49395f27f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:22.411588 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c24de-1e9c-408e-8dcb-6c49395f27f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:22.411883 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c24de-1e9c-408e-8dcb-6c49395f27f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:22.412168 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4c24de-1e9c-408e-8dcb-6c49395f27f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:22.412868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c24de-1e9c-408e-8dcb-6c49395f27f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:22.414817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4c24de-1e9c-408e-8dcb-6c49395f27f8 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:22.440167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 898/1778] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:01:22 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:22.444216 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bee012-c576-4342-a74b-e26ab65b59f8 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:22.455841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bee012-c576-4342-a74b-e26ab65b59f8 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:22.514774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21facf-cbde-4070-9133-dfa360324f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:22.515468 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21facf-cbde-4070-9133-dfa360324f33 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:22.515735 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21facf-cbde-4070-9133-dfa360324f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:22.516012 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21facf-cbde-4070-9133-dfa360324f33 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:22.517665 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21facf-cbde-4070-9133-dfa360324f33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:22.520012 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21facf-cbde-4070-9133-dfa360324f33 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:22.543187 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 881/1779] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:01:22 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:01:22.547078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c24de-1e9c-408e-8dcb-6c49395f27f8 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:22.579153 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4c24de-1e9c-408e-8dcb-6c49395f27f8 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:22.587425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9447d6-febb-4dc9-9cdb-9fb0b89c100c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:22.588896 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9447d6-febb-4dc9-9cdb-9fb0b89c100c admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:22.589284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9447d6-febb-4dc9-9cdb-9fb0b89c100c admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:22.590692 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9447d6-febb-4dc9-9cdb-9fb0b89c100c admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:22.592663 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9447d6-febb-4dc9-9cdb-9fb0b89c100c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:22.596112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9447d6-febb-4dc9-9cdb-9fb0b89c100c admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:22.615580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 899/1780] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:22.621404 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21facf-cbde-4070-9133-dfa360324f33 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:22.638108 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21facf-cbde-4070-9133-dfa360324f33 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:22.648016 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66042abd-275e-4fc8-9a61-40b4bd8b5288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:22.649543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66042abd-275e-4fc8-9a61-40b4bd8b5288 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:22.649995 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66042abd-275e-4fc8-9a61-40b4bd8b5288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:22.650402 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66042abd-275e-4fc8-9a61-40b4bd8b5288 admin admin] PATH_INFO: `/v3/projects/ece5d4850d724a268adddcf7a9e3f8b6/users/efc3e7991b9542b3a20a96d1950ab19f/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:22.677637 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66042abd-275e-4fc8-9a61-40b4bd8b5288 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ece5d4850d724a268adddcf7a9e3f8b6, user_id=efc3e7991b9542b3a20a96d1950ab19f, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:22.683122 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66042abd-275e-4fc8-9a61-40b4bd8b5288 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:22.714623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 882/1781] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:22 2022] PUT /identity/v3/projects/ece5d4850d724a268adddcf7a9e3f8b6/users/efc3e7991b9542b3a20a96d1950ab19f/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:22.721022 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9447d6-febb-4dc9-9cdb-9fb0b89c100c admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:22.742283 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9447d6-febb-4dc9-9cdb-9fb0b89c100c admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:22.877184 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ff692-5eb4-4096-ace8-a80501a30f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:22.880253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ff692-5eb4-4096-ace8-a80501a30f76 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:22.880547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ff692-5eb4-4096-ace8-a80501a30f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:22.880843 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ff692-5eb4-4096-ace8-a80501a30f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:22.883847 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ff692-5eb4-4096-ace8-a80501a30f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:22.885531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ff692-5eb4-4096-ace8-a80501a30f76 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:22.905179 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 900/1782] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:22 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:22.910016 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66042abd-275e-4fc8-9a61-40b4bd8b5288 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:22.927652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66042abd-275e-4fc8-9a61-40b4bd8b5288 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:22.940568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1eae66-3c9e-4f04-8250-80cd4ec241a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:22.941854 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1eae66-3c9e-4f04-8250-80cd4ec241a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:22.943226 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1eae66-3c9e-4f04-8250-80cd4ec241a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:22.943226 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1eae66-3c9e-4f04-8250-80cd4ec241a0 admin admin] PATH_INFO: `/v3/projects/ece5d4850d724a268adddcf7a9e3f8b6/users/efc3e7991b9542b3a20a96d1950ab19f/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:22.947952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1eae66-3c9e-4f04-8250-80cd4ec241a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ece5d4850d724a268adddcf7a9e3f8b6, user_id=efc3e7991b9542b3a20a96d1950ab19f, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:22.952597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1eae66-3c9e-4f04-8250-80cd4ec241a0 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:22.959349 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:22.985327 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 883/1783] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:22 2022] PUT /identity/v3/projects/ece5d4850d724a268adddcf7a9e3f8b6/users/efc3e7991b9542b3a20a96d1950ab19f/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:22.988265 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f9096-0933-49ea-9be1-b878446f961e None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:22.988265 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f9096-0933-49ea-9be1-b878446f961e None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:22.988265 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5f9096-0933-49ea-9be1-b878446f961e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:23.037364 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-9e5f9096-0933-49ea-9be1-b878446f961e None None] MFA Rules not processed for user `efc3e7991b9542b3a20a96d1950ab19f`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:23.038638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-9e5f9096-0933-49ea-9be1-b878446f961e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:23.055531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:23.241492 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 901/1784] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:22 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:23.250788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1eae66-3c9e-4f04-8250-80cd4ec241a0 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:23.254368 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-af1eae66-3c9e-4f04-8250-80cd4ec241a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:23.356015 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1eae66-3c9e-4f04-8250-80cd4ec241a0 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:23.360152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567c428-91f5-43f4-9d8f-336f34e7d612 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:23.361378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567c428-91f5-43f4-9d8f-336f34e7d612 service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:23.361667 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567c428-91f5-43f4-9d8f-336f34e7d612 service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:23.362580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567c428-91f5-43f4-9d8f-336f34e7d612 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:23.451636 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567c428-91f5-43f4-9d8f-336f34e7d612 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:23.454008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567c428-91f5-43f4-9d8f-336f34e7d612 service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:23.473008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 884/1785] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:23 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:31.589501 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f9096-0933-49ea-9be1-b878446f961e None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:31.662632 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5f9096-0933-49ea-9be1-b878446f961e None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:31.668344 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4512a74-04fe-4e91-b353-9d621caaa6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:31.669442 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4512a74-04fe-4e91-b353-9d621caaa6cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:31.669749 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4512a74-04fe-4e91-b353-9d621caaa6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:31.670014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4512a74-04fe-4e91-b353-9d621caaa6cb admin admin] PATH_INFO: `/v3/users/f53ab2860eea4264b469aea3da49e637` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:31.671276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4512a74-04fe-4e91-b353-9d621caaa6cb admin admin] RBAC: Authorizing `identity:delete_user(user_id=f53ab2860eea4264b469aea3da49e637)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:31.673344 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4512a74-04fe-4e91-b353-9d621caaa6cb admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:31.722632 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e4512a74-04fe-4e91-b353-9d621caaa6cb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f53ab2860eea4264b469aea3da49e637'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:31.731318 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e4512a74-04fe-4e91-b353-9d621caaa6cb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f53ab2860eea4264b469aea3da49e637'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:31.732086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e4512a74-04fe-4e91-b353-9d621caaa6cb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f53ab2860eea4264b469aea3da49e637'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:31.748238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e4512a74-04fe-4e91-b353-9d621caaa6cb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f53ab2860eea4264b469aea3da49e637'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:31.748950 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e4512a74-04fe-4e91-b353-9d621caaa6cb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f53ab2860eea4264b469aea3da49e637'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:31.764909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 902/1786] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:01:31 2022] DELETE /identity/v3/users/f53ab2860eea4264b469aea3da49e637 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:31.770678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f712c-4877-4e6d-be3b-fe807edc6f24 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:31.771208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f712c-4877-4e6d-be3b-fe807edc6f24 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:31.772321 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173f712c-4877-4e6d-be3b-fe807edc6f24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:31.797867 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-173f712c-4877-4e6d-be3b-fe807edc6f24 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:31.799628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-173f712c-4877-4e6d-be3b-fe807edc6f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:31.875114 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 885/1787] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:31.887808 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4512a74-04fe-4e91-b353-9d621caaa6cb admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:31.889530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e4512a74-04fe-4e91-b353-9d621caaa6cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:31.927366 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:31.984095 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4512a74-04fe-4e91-b353-9d621caaa6cb admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:32.012430 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7cc3c-d889-4537-8268-5768db0d77e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:32.013114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7cc3c-d889-4537-8268-5768db0d77e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:32.013405 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7cc3c-d889-4537-8268-5768db0d77e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:32.013689 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee7cc3c-d889-4537-8268-5768db0d77e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:32.027171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee7cc3c-d889-4537-8268-5768db0d77e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:32.028675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee7cc3c-d889-4537-8268-5768db0d77e0 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:32.045632 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 903/1788] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:32.524303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f712c-4877-4e6d-be3b-fe807edc6f24 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:32.525964 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-173f712c-4877-4e6d-be3b-fe807edc6f24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:32.559749 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-173f712c-4877-4e6d-be3b-fe807edc6f24 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:32.582767 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e596207e-7fcb-48c7-9ed2-10d909d452c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:32.583841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596207e-7fcb-48c7-9ed2-10d909d452c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:32.586760 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596207e-7fcb-48c7-9ed2-10d909d452c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:32.586760 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e596207e-7fcb-48c7-9ed2-10d909d452c1 admin admin] PATH_INFO: `/v3/users/9e99ede07e7e4622841148ac56398c31` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:32.586760 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596207e-7fcb-48c7-9ed2-10d909d452c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e99ede07e7e4622841148ac56398c31)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:32.598007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e596207e-7fcb-48c7-9ed2-10d909d452c1 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:32.709815 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:INSERT=1,keystone:SELECT=93,keystone:DELETE=6 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:32.716834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-e596207e-7fcb-48c7-9ed2-10d909d452c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e99ede07e7e4622841148ac56398c31'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:32.717579 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-e596207e-7fcb-48c7-9ed2-10d909d452c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e99ede07e7e4622841148ac56398c31'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:32.732916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-e596207e-7fcb-48c7-9ed2-10d909d452c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e99ede07e7e4622841148ac56398c31'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:32.736002 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-e596207e-7fcb-48c7-9ed2-10d909d452c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e99ede07e7e4622841148ac56398c31'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:32.752035 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-e596207e-7fcb-48c7-9ed2-10d909d452c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e99ede07e7e4622841148ac56398c31'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:32.754138 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7cc3c-d889-4537-8268-5768db0d77e0 service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:32.755904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7ee7cc3c-d889-4537-8268-5768db0d77e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:32.775265 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 886/1789] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:01:32 2022] DELETE /identity/v3/users/9e99ede07e7e4622841148ac56398c31 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:32.788206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0ca67-d11c-4e3c-989f-f2b1e0b3450f None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:32.788509 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0ca67-d11c-4e3c-989f-f2b1e0b3450f None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:32.788796 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0ca67-d11c-4e3c-989f-f2b1e0b3450f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:32.793486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee7cc3c-d889-4537-8268-5768db0d77e0 service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:32.818946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-72f0ca67-d11c-4e3c-989f-f2b1e0b3450f None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:32.820445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-72f0ca67-d11c-4e3c-989f-f2b1e0b3450f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:32.916127 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 887/1790] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:32.929560 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0ca67-d11c-4e3c-989f-f2b1e0b3450f None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:32.931493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-72f0ca67-d11c-4e3c-989f-f2b1e0b3450f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:32.948473 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f2c23-46d5-40c0-8851-9f7fb177a729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:32.949999 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f2c23-46d5-40c0-8851-9f7fb177a729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:32.950534 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f2c23-46d5-40c0-8851-9f7fb177a729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:32.951100 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54f2c23-46d5-40c0-8851-9f7fb177a729 admin admin] PATH_INFO: `/v3/projects/4246f6446ef14446bf754b904f2e685a` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:32.952458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f2c23-46d5-40c0-8851-9f7fb177a729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4246f6446ef14446bf754b904f2e685a)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:32.955000 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54f2c23-46d5-40c0-8851-9f7fb177a729 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:33.007229 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e54f2c23-46d5-40c0-8851-9f7fb177a729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4246f6446ef14446bf754b904f2e685a', 'request_id': 'req-e54f2c23-46d5-40c0-8851-9f7fb177a729'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.014185 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e54f2c23-46d5-40c0-8851-9f7fb177a729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4246f6446ef14446bf754b904f2e685a', 'request_id': 'req-e54f2c23-46d5-40c0-8851-9f7fb177a729'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.015027 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0ca67-d11c-4e3c-989f-f2b1e0b3450f None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:33.020618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e54f2c23-46d5-40c0-8851-9f7fb177a729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4246f6446ef14446bf754b904f2e685a', 'request_id': 'req-e54f2c23-46d5-40c0-8851-9f7fb177a729'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.033050 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e54f2c23-46d5-40c0-8851-9f7fb177a729 admin admin] The token cache is being invalidate because project 4246f6446ef14446bf754b904f2e685a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:01:33.033429 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e54f2c23-46d5-40c0-8851-9f7fb177a729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.034073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e54f2c23-46d5-40c0-8851-9f7fb177a729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.035862 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 904/1791] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:01:32 2022] DELETE /identity/v3/projects/4246f6446ef14446bf754b904f2e685a => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:33.039426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f2c23-46d5-40c0-8851-9f7fb177a729 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:33.040770 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e54f2c23-46d5-40c0-8851-9f7fb177a729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:33.070395 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd29465-6ee3-4c20-89df-847021402bb9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:33.071351 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd29465-6ee3-4c20-89df-847021402bb9 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:33.071717 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd29465-6ee3-4c20-89df-847021402bb9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:33.072213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd29465-6ee3-4c20-89df-847021402bb9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:33.074148 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-7bd29465-6ee3-4c20-89df-847021402bb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:33.093555 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:33.107177 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:33.109523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54f2c23-46d5-40c0-8851-9f7fb177a729 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:33.115087 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f149603-9d99-465f-8357-2e9efb762153 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:33.116022 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f149603-9d99-465f-8357-2e9efb762153 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:33.116334 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f149603-9d99-465f-8357-2e9efb762153 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:33.116822 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f149603-9d99-465f-8357-2e9efb762153 admin admin] PATH_INFO: `/v3/users/efc3e7991b9542b3a20a96d1950ab19f` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:33.117936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f149603-9d99-465f-8357-2e9efb762153 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efc3e7991b9542b3a20a96d1950ab19f)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:33.119898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f149603-9d99-465f-8357-2e9efb762153 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:33.142456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd29465-6ee3-4c20-89df-847021402bb9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:33.143973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd29465-6ee3-4c20-89df-847021402bb9 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:33.155200 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 888/1792] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:01:33.170979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5f149603-9d99-465f-8357-2e9efb762153 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efc3e7991b9542b3a20a96d1950ab19f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.178684 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5f149603-9d99-465f-8357-2e9efb762153 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efc3e7991b9542b3a20a96d1950ab19f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.179405 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5f149603-9d99-465f-8357-2e9efb762153 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efc3e7991b9542b3a20a96d1950ab19f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.193791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5f149603-9d99-465f-8357-2e9efb762153 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efc3e7991b9542b3a20a96d1950ab19f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.194547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5f149603-9d99-465f-8357-2e9efb762153 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efc3e7991b9542b3a20a96d1950ab19f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.208339 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 905/1793] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:01:33 2022] DELETE /identity/v3/users/efc3e7991b9542b3a20a96d1950ab19f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:01:33.439990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd29465-6ee3-4c20-89df-847021402bb9 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:33.441529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-7bd29465-6ee3-4c20-89df-847021402bb9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:33.456418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd29465-6ee3-4c20-89df-847021402bb9 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:33.526478 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222a450-78e8-4291-908f-cc35897a938b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:33.526675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f149603-9d99-465f-8357-2e9efb762153 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:33.527683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222a450-78e8-4291-908f-cc35897a938b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:33.528065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222a450-78e8-4291-908f-cc35897a938b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:33.528233 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5f149603-9d99-465f-8357-2e9efb762153 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:33.528519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5222a450-78e8-4291-908f-cc35897a938b admin admin] PATH_INFO: `/v3/projects/0701abb69e2f4f3d95a115e4212daa66` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:33.529819 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222a450-78e8-4291-908f-cc35897a938b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0701abb69e2f4f3d95a115e4212daa66)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:33.532455 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5222a450-78e8-4291-908f-cc35897a938b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:33.543453 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f149603-9d99-465f-8357-2e9efb762153 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:33.549085 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6943a7-e6b3-4447-a076-a05add994c8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:33.550080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6943a7-e6b3-4447-a076-a05add994c8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:33.550371 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6943a7-e6b3-4447-a076-a05add994c8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:33.550682 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6943a7-e6b3-4447-a076-a05add994c8f admin admin] PATH_INFO: `/v3/projects/ece5d4850d724a268adddcf7a9e3f8b6` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:33.551976 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6943a7-e6b3-4447-a076-a05add994c8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ece5d4850d724a268adddcf7a9e3f8b6)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:33.554071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6943a7-e6b3-4447-a076-a05add994c8f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:33.603769 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5222a450-78e8-4291-908f-cc35897a938b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0701abb69e2f4f3d95a115e4212daa66', 'request_id': 'req-5222a450-78e8-4291-908f-cc35897a938b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.624983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5222a450-78e8-4291-908f-cc35897a938b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0701abb69e2f4f3d95a115e4212daa66', 'request_id': 'req-5222a450-78e8-4291-908f-cc35897a938b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.626809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1a6943a7-e6b3-4447-a076-a05add994c8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ece5d4850d724a268adddcf7a9e3f8b6', 'request_id': 'req-1a6943a7-e6b3-4447-a076-a05add994c8f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.631632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5222a450-78e8-4291-908f-cc35897a938b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0701abb69e2f4f3d95a115e4212daa66', 'request_id': 'req-5222a450-78e8-4291-908f-cc35897a938b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.637023 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1a6943a7-e6b3-4447-a076-a05add994c8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ece5d4850d724a268adddcf7a9e3f8b6', 'request_id': 'req-1a6943a7-e6b3-4447-a076-a05add994c8f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.647612 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1a6943a7-e6b3-4447-a076-a05add994c8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ece5d4850d724a268adddcf7a9e3f8b6', 'request_id': 'req-1a6943a7-e6b3-4447-a076-a05add994c8f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.654846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5222a450-78e8-4291-908f-cc35897a938b admin admin] The token cache is being invalidate because project 0701abb69e2f4f3d95a115e4212daa66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:01:33.655538 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5222a450-78e8-4291-908f-cc35897a938b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.656729 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5222a450-78e8-4291-908f-cc35897a938b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.659650 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 889/1794] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:01:33 2022] DELETE /identity/v3/projects/0701abb69e2f4f3d95a115e4212daa66 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:33.664875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222a450-78e8-4291-908f-cc35897a938b admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:33.667444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-5222a450-78e8-4291-908f-cc35897a938b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:33.685637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1a6943a7-e6b3-4447-a076-a05add994c8f admin admin] The token cache is being invalidate because project ece5d4850d724a268adddcf7a9e3f8b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:01:33.685637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1a6943a7-e6b3-4447-a076-a05add994c8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.685637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1a6943a7-e6b3-4447-a076-a05add994c8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.685637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 906/1795] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:01:33 2022] DELETE /identity/v3/projects/ece5d4850d724a268adddcf7a9e3f8b6 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:33.691262 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1a08b-bfeb-4a51-a008-8bc9cb85848d None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:33.691598 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1a08b-bfeb-4a51-a008-8bc9cb85848d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:33.691878 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b1a08b-bfeb-4a51-a008-8bc9cb85848d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:33.713002 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-24b1a08b-bfeb-4a51-a008-8bc9cb85848d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:33.714264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-24b1a08b-bfeb-4a51-a008-8bc9cb85848d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:33.756484 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:33.761318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5222a450-78e8-4291-908f-cc35897a938b admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:33.768674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:33.792940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 907/1796] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:01:33.794716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93f764-9071-4942-b99e-e757f3c6be91 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:33.795020 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93f764-9071-4942-b99e-e757f3c6be91 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:33.795347 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b93f764-9071-4942-b99e-e757f3c6be91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:33.820066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-8b93f764-9071-4942-b99e-e757f3c6be91 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:33.821762 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-8b93f764-9071-4942-b99e-e757f3c6be91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:33.838016 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946f10a-1dfb-4d4d-8f5c-c2f0df7a2095 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:33.839062 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946f10a-1dfb-4d4d-8f5c-c2f0df7a2095 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:33.839431 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946f10a-1dfb-4d4d-8f5c-c2f0df7a2095 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:33.839821 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b946f10a-1dfb-4d4d-8f5c-c2f0df7a2095 admin admin] PATH_INFO: `/v3/users/95733192f45d43b89b1f2e3ac9a5f14e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:33.841256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946f10a-1dfb-4d4d-8f5c-c2f0df7a2095 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95733192f45d43b89b1f2e3ac9a5f14e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:33.845313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 908/1797] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:33.845467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b946f10a-1dfb-4d4d-8f5c-c2f0df7a2095 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:33.847081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93f764-9071-4942-b99e-e757f3c6be91 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:33.860750 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b93f764-9071-4942-b99e-e757f3c6be91 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:33.867807 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ae113-2aaa-42c5-acea-6e44518bf9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:33.868558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ae113-2aaa-42c5-acea-6e44518bf9b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:33.869090 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ae113-2aaa-42c5-acea-6e44518bf9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:33.869435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ae113-2aaa-42c5-acea-6e44518bf9b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:33.876269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ae113-2aaa-42c5-acea-6e44518bf9b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:01:33.876794 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ae113-2aaa-42c5-acea-6e44518bf9b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:33.878784 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61ae113-2aaa-42c5-acea-6e44518bf9b9 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:33.895902 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 909/1798] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:01:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:01:33.900519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ae113-2aaa-42c5-acea-6e44518bf9b9 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:33.914193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ae113-2aaa-42c5-acea-6e44518bf9b9 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:33.918321 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b946f10a-1dfb-4d4d-8f5c-c2f0df7a2095 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95733192f45d43b89b1f2e3ac9a5f14e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.919011 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b946f10a-1dfb-4d4d-8f5c-c2f0df7a2095 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95733192f45d43b89b1f2e3ac9a5f14e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.920967 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f05fd9-a49f-471a-b8c4-d794eae032e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:33.921722 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f05fd9-a49f-471a-b8c4-d794eae032e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:33.923327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f05fd9-a49f-471a-b8c4-d794eae032e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:33.923624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f05fd9-a49f-471a-b8c4-d794eae032e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:33.924329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f05fd9-a49f-471a-b8c4-d794eae032e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:33.926347 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f05fd9-a49f-471a-b8c4-d794eae032e5 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:33.927725 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b946f10a-1dfb-4d4d-8f5c-c2f0df7a2095 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95733192f45d43b89b1f2e3ac9a5f14e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.928473 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b946f10a-1dfb-4d4d-8f5c-c2f0df7a2095 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95733192f45d43b89b1f2e3ac9a5f14e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.942673 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b946f10a-1dfb-4d4d-8f5c-c2f0df7a2095 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95733192f45d43b89b1f2e3ac9a5f14e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:33.957185 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 910/1799] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:01:33 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:01:33.957730 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 890/1800] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:01:33 2022] DELETE /identity/v3/users/95733192f45d43b89b1f2e3ac9a5f14e => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:33.962484 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f05fd9-a49f-471a-b8c4-d794eae032e5 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:33.964355 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a1f05fd9-a49f-471a-b8c4-d794eae032e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:34.040980 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f05fd9-a49f-471a-b8c4-d794eae032e5 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:34.116660 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3e085-0f70-4f10-9e1e-d465787310ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:34.117443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3e085-0f70-4f10-9e1e-d465787310ec admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:34.117829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3e085-0f70-4f10-9e1e-d465787310ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:34.118198 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b3e085-0f70-4f10-9e1e-d465787310ec admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:34.119097 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3e085-0f70-4f10-9e1e-d465787310ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:34.121140 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b3e085-0f70-4f10-9e1e-d465787310ec admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:34.157462 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 911/1801] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:01:33 2022] POST /identity/v3/users => generated 287 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:34.157771 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946f10a-1dfb-4d4d-8f5c-c2f0df7a2095 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:34.168572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b946f10a-1dfb-4d4d-8f5c-c2f0df7a2095 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:34.174038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b5775-8b66-4bc5-bc7a-152701d0c233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:34.174926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b5775-8b66-4bc5-bc7a-152701d0c233 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:34.175288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b5775-8b66-4bc5-bc7a-152701d0c233 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:34.175652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711b5775-8b66-4bc5-bc7a-152701d0c233 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:34.176581 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b5775-8b66-4bc5-bc7a-152701d0c233 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:34.179065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711b5775-8b66-4bc5-bc7a-152701d0c233 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:34.194650 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 891/1802] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:01:34.199093 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3e085-0f70-4f10-9e1e-d465787310ec admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:34.217868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b3e085-0f70-4f10-9e1e-d465787310ec admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:34.230824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f7acb-6fda-49ca-bec9-9bee7a3e02a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:34.230824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f7acb-6fda-49ca-bec9-9bee7a3e02a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:34.230824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f7acb-6fda-49ca-bec9-9bee7a3e02a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:34.230824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918f7acb-6fda-49ca-bec9-9bee7a3e02a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:34.231174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f7acb-6fda-49ca-bec9-9bee7a3e02a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:34.231174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918f7acb-6fda-49ca-bec9-9bee7a3e02a9 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:34.245467 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 912/1803] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:34.249451 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b5775-8b66-4bc5-bc7a-152701d0c233 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:34.262615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711b5775-8b66-4bc5-bc7a-152701d0c233 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:34.264789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f7acb-6fda-49ca-bec9-9bee7a3e02a9 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:34.266648 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-918f7acb-6fda-49ca-bec9-9bee7a3e02a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:34.272505 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f140719c-7c0e-4fa7-8ada-65f0c33e98b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:34.274538 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140719c-7c0e-4fa7-8ada-65f0c33e98b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:34.275875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140719c-7c0e-4fa7-8ada-65f0c33e98b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:34.276760 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f140719c-7c0e-4fa7-8ada-65f0c33e98b3 admin admin] PATH_INFO: `/v3/projects/b30b90ec4d5e45468a4fa0fb98e46393/users/01dd0e5661f44d05a01d7a5e82dbadf3/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:34.294877 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918f7acb-6fda-49ca-bec9-9bee7a3e02a9 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:34.300029 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140719c-7c0e-4fa7-8ada-65f0c33e98b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b30b90ec4d5e45468a4fa0fb98e46393, user_id=01dd0e5661f44d05a01d7a5e82dbadf3, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:34.300955 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4e4cb-a745-432e-887e-48ec5a260292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:34.301960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4e4cb-a745-432e-887e-48ec5a260292 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:34.302440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4e4cb-a745-432e-887e-48ec5a260292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:34.302928 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f4e4cb-a745-432e-887e-48ec5a260292 admin admin] PATH_INFO: `/v3/projects/ed0a26b2ea754016b1fc625c1e6d8bc8` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:34.303153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f140719c-7c0e-4fa7-8ada-65f0c33e98b3 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:34.304590 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4e4cb-a745-432e-887e-48ec5a260292 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed0a26b2ea754016b1fc625c1e6d8bc8)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:34.307066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f4e4cb-a745-432e-887e-48ec5a260292 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:34.322640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 892/1804] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:34 2022] PUT /identity/v3/projects/b30b90ec4d5e45468a4fa0fb98e46393/users/01dd0e5661f44d05a01d7a5e82dbadf3/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:34.327512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b1977-b241-447c-8100-260ccb1678b1 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:34.327810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b1977-b241-447c-8100-260ccb1678b1 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:34.328084 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b1977-b241-447c-8100-260ccb1678b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:34.347438 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-dc4b1977-b241-447c-8100-260ccb1678b1 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:34.348565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-dc4b1977-b241-447c-8100-260ccb1678b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:34.364830 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:34.385306 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-b2f4e4cb-a745-432e-887e-48ec5a260292 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed0a26b2ea754016b1fc625c1e6d8bc8', 'request_id': 'req-b2f4e4cb-a745-432e-887e-48ec5a260292'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:34.395513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-b2f4e4cb-a745-432e-887e-48ec5a260292 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed0a26b2ea754016b1fc625c1e6d8bc8', 'request_id': 'req-b2f4e4cb-a745-432e-887e-48ec5a260292'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:34.400757 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:34.403426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-b2f4e4cb-a745-432e-887e-48ec5a260292 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed0a26b2ea754016b1fc625c1e6d8bc8', 'request_id': 'req-b2f4e4cb-a745-432e-887e-48ec5a260292'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:34.419709 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-b2f4e4cb-a745-432e-887e-48ec5a260292 admin admin] The token cache is being invalidate because project ed0a26b2ea754016b1fc625c1e6d8bc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:01:34.423423 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-b2f4e4cb-a745-432e-887e-48ec5a260292 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:34.423745 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 893/1805] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:01:34.423835 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-b2f4e4cb-a745-432e-887e-48ec5a260292 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:34.423835 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 913/1806] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:01:34 2022] DELETE /identity/v3/projects/ed0a26b2ea754016b1fc625c1e6d8bc8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:01:34.428528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95351514-d1b9-4787-9ba0-71db27200c89 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:34.428852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95351514-d1b9-4787-9ba0-71db27200c89 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:34.429175 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95351514-d1b9-4787-9ba0-71db27200c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:34.431389 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4e4cb-a745-432e-887e-48ec5a260292 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:34.433245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b2f4e4cb-a745-432e-887e-48ec5a260292 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:34.452086 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-95351514-d1b9-4787-9ba0-71db27200c89 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:34.453849 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-95351514-d1b9-4787-9ba0-71db27200c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:34.496248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f4e4cb-a745-432e-887e-48ec5a260292 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:34.500154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64051cff-ab5c-488c-a985-327f3502adb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:34.500900 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64051cff-ab5c-488c-a985-327f3502adb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:34.501267 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64051cff-ab5c-488c-a985-327f3502adb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:34.501625 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64051cff-ab5c-488c-a985-327f3502adb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:34.504534 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-64051cff-ab5c-488c-a985-327f3502adb4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:34.518118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 894/1807] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:01:34.525648 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb952da-aec5-4e2a-908e-d84c6d623733 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:34.525947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb952da-aec5-4e2a-908e-d84c6d623733 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:34.526231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb952da-aec5-4e2a-908e-d84c6d623733 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:34.544942 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-2fb952da-aec5-4e2a-908e-d84c6d623733 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:34.546537 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-2fb952da-aec5-4e2a-908e-d84c6d623733 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:34.554986 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 895/1808] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:01:34.559349 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb952da-aec5-4e2a-908e-d84c6d623733 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:34.569830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb952da-aec5-4e2a-908e-d84c6d623733 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:34.574673 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117711e8-73e6-41e9-bbdf-78627a0849dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:34.575341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117711e8-73e6-41e9-bbdf-78627a0849dc admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:34.575624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117711e8-73e6-41e9-bbdf-78627a0849dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:34.575901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-117711e8-73e6-41e9-bbdf-78627a0849dc admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:34.576670 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117711e8-73e6-41e9-bbdf-78627a0849dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:01:34.576981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117711e8-73e6-41e9-bbdf-78627a0849dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:34.578736 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-117711e8-73e6-41e9-bbdf-78627a0849dc admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:34.583991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64051cff-ab5c-488c-a985-327f3502adb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:34.585537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64051cff-ab5c-488c-a985-327f3502adb4 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:34.595229 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 896/1809] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:01:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:01:34.596035 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 914/1810] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:34.597823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117711e8-73e6-41e9-bbdf-78627a0849dc admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:34.607939 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-117711e8-73e6-41e9-bbdf-78627a0849dc admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:34.613096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e9fd7-43c1-477c-be06-71f454ac080c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:34.613753 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e9fd7-43c1-477c-be06-71f454ac080c admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:34.614037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e9fd7-43c1-477c-be06-71f454ac080c admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:34.614316 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e9fd7-43c1-477c-be06-71f454ac080c admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:34.615045 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e9fd7-43c1-477c-be06-71f454ac080c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:34.617528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e9fd7-43c1-477c-be06-71f454ac080c admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:34.660428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 897/1811] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:01:34 2022] POST /identity/v3/projects => generated 389 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:34.661557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64051cff-ab5c-488c-a985-327f3502adb4 service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:34.671953 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64051cff-ab5c-488c-a985-327f3502adb4 service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:34.744207 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc11c1-71e4-4afb-b83b-535fade908ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:34.745048 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc11c1-71e4-4afb-b83b-535fade908ee admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:34.745450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc11c1-71e4-4afb-b83b-535fade908ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:34.745804 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbc11c1-71e4-4afb-b83b-535fade908ee admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:34.746665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc11c1-71e4-4afb-b83b-535fade908ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:34.749030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbc11c1-71e4-4afb-b83b-535fade908ee admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:34.775201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 915/1812] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:01:34 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:34.778718 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e9fd7-43c1-477c-be06-71f454ac080c admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:34.792563 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e9fd7-43c1-477c-be06-71f454ac080c admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:34.798060 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba44fdd-cf0f-4366-8a6f-f150fa98a84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:34.798795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba44fdd-cf0f-4366-8a6f-f150fa98a84b admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:34.799083 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba44fdd-cf0f-4366-8a6f-f150fa98a84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:34.799359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba44fdd-cf0f-4366-8a6f-f150fa98a84b admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:34.800089 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba44fdd-cf0f-4366-8a6f-f150fa98a84b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:34.802867 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba44fdd-cf0f-4366-8a6f-f150fa98a84b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:34.815529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 898/1813] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:34.819680 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc11c1-71e4-4afb-b83b-535fade908ee admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:34.827918 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:34.837738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbc11c1-71e4-4afb-b83b-535fade908ee admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:34.846758 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533e3597-c89b-4951-92d9-d25c9a945a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:34.847567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e3597-c89b-4951-92d9-d25c9a945a23 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:34.848090 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e3597-c89b-4951-92d9-d25c9a945a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:34.848199 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533e3597-c89b-4951-92d9-d25c9a945a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:34.849018 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e3597-c89b-4951-92d9-d25c9a945a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:34.850743 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533e3597-c89b-4951-92d9-d25c9a945a23 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:34.866038 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 916/1814] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:34 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:34.867343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba44fdd-cf0f-4366-8a6f-f150fa98a84b admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:34.879508 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba44fdd-cf0f-4366-8a6f-f150fa98a84b admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:34.886670 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e860c-9aec-4b9e-92ed-45fbec75c227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:34.887689 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e860c-9aec-4b9e-92ed-45fbec75c227 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:34.888621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e860c-9aec-4b9e-92ed-45fbec75c227 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:34.888621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99e860c-9aec-4b9e-92ed-45fbec75c227 admin admin] PATH_INFO: `/v3/projects/b1f464b3d677429a8239cd1fb6100d49/users/dd115dce841d485d986a36f31a72b30b/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:34.905326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e860c-9aec-4b9e-92ed-45fbec75c227 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1f464b3d677429a8239cd1fb6100d49, user_id=dd115dce841d485d986a36f31a72b30b, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:34.908101 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99e860c-9aec-4b9e-92ed-45fbec75c227 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:34.921236 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 899/1815] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:34 2022] PUT /identity/v3/projects/b1f464b3d677429a8239cd1fb6100d49/users/dd115dce841d485d986a36f31a72b30b/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:34.932815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf8d7d-20e2-4872-b537-c04e5b587cfe None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:34.933205 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf8d7d-20e2-4872-b537-c04e5b587cfe None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:34.933572 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdf8d7d-20e2-4872-b537-c04e5b587cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:34.965169 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-5bdf8d7d-20e2-4872-b537-c04e5b587cfe None None] MFA Rules not processed for user `dd115dce841d485d986a36f31a72b30b`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:34.967138 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5bdf8d7d-20e2-4872-b537-c04e5b587cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:35.051766 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 917/1816] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:35.056395 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e860c-9aec-4b9e-92ed-45fbec75c227 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:35.120930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99e860c-9aec-4b9e-92ed-45fbec75c227 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:35.126273 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa2783-214d-4dcb-8824-bba69d7aa6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:35.126985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa2783-214d-4dcb-8824-bba69d7aa6ae admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:35.127278 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa2783-214d-4dcb-8824-bba69d7aa6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:35.127571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefa2783-214d-4dcb-8824-bba69d7aa6ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:35.128304 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa2783-214d-4dcb-8824-bba69d7aa6ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:35.130230 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefa2783-214d-4dcb-8824-bba69d7aa6ae admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:35.152937 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:35.161251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 900/1817] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:01:35 2022] POST /identity/v3/projects => generated 389 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:01:35.165268 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf8d7d-20e2-4872-b537-c04e5b587cfe None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:35.176907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdf8d7d-20e2-4872-b537-c04e5b587cfe None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:35.242382 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ced1c-8645-4126-8f12-f0dcc60992aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:35.243103 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ced1c-8645-4126-8f12-f0dcc60992aa admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:35.243402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ced1c-8645-4126-8f12-f0dcc60992aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:35.243692 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ced1c-8645-4126-8f12-f0dcc60992aa admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:35.244397 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ced1c-8645-4126-8f12-f0dcc60992aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:35.246480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ced1c-8645-4126-8f12-f0dcc60992aa admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:35.274250 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 918/1818] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:01:35 2022] POST /identity/v3/users => generated 301 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:01:35.277870 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa2783-214d-4dcb-8824-bba69d7aa6ae admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:35.293823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefa2783-214d-4dcb-8824-bba69d7aa6ae admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:35.299598 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0be61-9701-4f63-89dc-633e83cf97c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:35.300315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0be61-9701-4f63-89dc-633e83cf97c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:35.301004 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0be61-9701-4f63-89dc-633e83cf97c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:35.301288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e0be61-9701-4f63-89dc-633e83cf97c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:35.302047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0be61-9701-4f63-89dc-633e83cf97c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:35.303667 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e0be61-9701-4f63-89dc-633e83cf97c9 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:35.316712 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 901/1819] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:35.320912 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ced1c-8645-4126-8f12-f0dcc60992aa admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:35.332320 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ced1c-8645-4126-8f12-f0dcc60992aa admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:35.337307 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51843ab-11c1-41d6-8caa-9681d6d73f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:35.338228 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51843ab-11c1-41d6-8caa-9681d6d73f94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:35.338523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51843ab-11c1-41d6-8caa-9681d6d73f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:35.338837 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51843ab-11c1-41d6-8caa-9681d6d73f94 admin admin] PATH_INFO: `/v3/projects/9667137f74dd4b1ab149c1dabc24ec22/users/f4910fe8b73c4d5589dfe96c880a0c9e/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:35.355027 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51843ab-11c1-41d6-8caa-9681d6d73f94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9667137f74dd4b1ab149c1dabc24ec22, user_id=f4910fe8b73c4d5589dfe96c880a0c9e, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:35.358060 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51843ab-11c1-41d6-8caa-9681d6d73f94 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:35.374629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 919/1820] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:35 2022] PUT /identity/v3/projects/9667137f74dd4b1ab149c1dabc24ec22/users/f4910fe8b73c4d5589dfe96c880a0c9e/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:01:35.378141 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0be61-9701-4f63-89dc-633e83cf97c9 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:35.391322 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e0be61-9701-4f63-89dc-633e83cf97c9 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:35.454584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae4aa81-4ae4-4473-9495-4d4e787e105a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:35.455394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4aa81-4ae4-4473-9495-4d4e787e105a admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:35.455776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4aa81-4ae4-4473-9495-4d4e787e105a admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:35.456233 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae4aa81-4ae4-4473-9495-4d4e787e105a admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:35.457026 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4aa81-4ae4-4473-9495-4d4e787e105a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:35.458760 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae4aa81-4ae4-4473-9495-4d4e787e105a admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:35.473170 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 902/1821] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:35 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:35.474328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51843ab-11c1-41d6-8caa-9681d6d73f94 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:35.485659 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51843ab-11c1-41d6-8caa-9681d6d73f94 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:35.493088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8303a56-fbce-4137-a8ee-629c3de6228c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:35.494333 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8303a56-fbce-4137-a8ee-629c3de6228c admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:35.494732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8303a56-fbce-4137-a8ee-629c3de6228c admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:35.495025 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8303a56-fbce-4137-a8ee-629c3de6228c admin admin] PATH_INFO: `/v3/projects/9667137f74dd4b1ab149c1dabc24ec22/users/f4910fe8b73c4d5589dfe96c880a0c9e/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:35.497230 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8303a56-fbce-4137-a8ee-629c3de6228c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9667137f74dd4b1ab149c1dabc24ec22, user_id=f4910fe8b73c4d5589dfe96c880a0c9e, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:35.499987 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8303a56-fbce-4137-a8ee-629c3de6228c admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:35.521364 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 920/1822] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:35 2022] PUT /identity/v3/projects/9667137f74dd4b1ab149c1dabc24ec22/users/f4910fe8b73c4d5589dfe96c880a0c9e/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:35.532899 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a426f-9c78-4677-83af-afafa856e16b None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:35.533130 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a426f-9c78-4677-83af-afafa856e16b None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:35.533419 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883a426f-9c78-4677-83af-afafa856e16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:35.563835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-883a426f-9c78-4677-83af-afafa856e16b None None] MFA Rules not processed for user `f4910fe8b73c4d5589dfe96c880a0c9e`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:35.565155 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-883a426f-9c78-4677-83af-afafa856e16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:35.646546 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 903/1823] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:35 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:35.663386 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8303a56-fbce-4137-a8ee-629c3de6228c admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:35.666836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-d8303a56-fbce-4137-a8ee-629c3de6228c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:35.713719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:35.741563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8303a56-fbce-4137-a8ee-629c3de6228c admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:35.745562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b80708e-6081-4607-95e3-0a88215c4fed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:35.746231 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80708e-6081-4607-95e3-0a88215c4fed service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:35.746522 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80708e-6081-4607-95e3-0a88215c4fed service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:35.746838 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b80708e-6081-4607-95e3-0a88215c4fed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:35.757889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80708e-6081-4607-95e3-0a88215c4fed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:35.759423 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b80708e-6081-4607-95e3-0a88215c4fed service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:35.772655 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 921/1824] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:35 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:35.930297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a426f-9c78-4677-83af-afafa856e16b None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:36.029422 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883a426f-9c78-4677-83af-afafa856e16b None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:36.035439 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876e3af-1ed8-4173-9aec-22853d48bf0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:36.036606 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876e3af-1ed8-4173-9aec-22853d48bf0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:36.036984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876e3af-1ed8-4173-9aec-22853d48bf0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:36.037459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4876e3af-1ed8-4173-9aec-22853d48bf0c admin admin] PATH_INFO: `/v3/users/dd115dce841d485d986a36f31a72b30b` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:36.038940 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876e3af-1ed8-4173-9aec-22853d48bf0c admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd115dce841d485d986a36f31a72b30b)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:36.041231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4876e3af-1ed8-4173-9aec-22853d48bf0c admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:36.055550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:36.106246 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-4876e3af-1ed8-4173-9aec-22853d48bf0c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd115dce841d485d986a36f31a72b30b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:36.107110 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-4876e3af-1ed8-4173-9aec-22853d48bf0c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd115dce841d485d986a36f31a72b30b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:36.115204 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-4876e3af-1ed8-4173-9aec-22853d48bf0c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd115dce841d485d986a36f31a72b30b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:36.115896 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-4876e3af-1ed8-4173-9aec-22853d48bf0c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd115dce841d485d986a36f31a72b30b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:36.136249 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-4876e3af-1ed8-4173-9aec-22853d48bf0c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd115dce841d485d986a36f31a72b30b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:36.161034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 904/1825] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:01:35 2022] DELETE /identity/v3/users/dd115dce841d485d986a36f31a72b30b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:36.165445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab699d-b4ba-4d3f-9391-d29e6a195b90 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:36.165907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab699d-b4ba-4d3f-9391-d29e6a195b90 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:36.166076 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ab699d-b4ba-4d3f-9391-d29e6a195b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:36.193309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-55ab699d-b4ba-4d3f-9391-d29e6a195b90 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:36.194647 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-55ab699d-b4ba-4d3f-9391-d29e6a195b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:36.278470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 922/1826] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:36.292319 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876e3af-1ed8-4173-9aec-22853d48bf0c admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:36.295453 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-4876e3af-1ed8-4173-9aec-22853d48bf0c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:36.441984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4876e3af-1ed8-4173-9aec-22853d48bf0c admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:36.446196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf2915-9819-4a80-9ff9-bc9611b4c5ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:36.446894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf2915-9819-4a80-9ff9-bc9611b4c5ff service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:36.447188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf2915-9819-4a80-9ff9-bc9611b4c5ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:36.447471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf2915-9819-4a80-9ff9-bc9611b4c5ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:36.460662 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf2915-9819-4a80-9ff9-bc9611b4c5ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:36.471873 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf2915-9819-4a80-9ff9-bc9611b4c5ff service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:36.488888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 905/1827] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:38.126297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab699d-b4ba-4d3f-9391-d29e6a195b90 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:38.127782 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-55ab699d-b4ba-4d3f-9391-d29e6a195b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:38.147687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ab699d-b4ba-4d3f-9391-d29e6a195b90 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:38.153005 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468141a-7ca1-4f50-b32c-34f88f0a0cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:38.153904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468141a-7ca1-4f50-b32c-34f88f0a0cb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:38.154325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468141a-7ca1-4f50-b32c-34f88f0a0cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:38.154646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9468141a-7ca1-4f50-b32c-34f88f0a0cb7 admin admin] PATH_INFO: `/v3/projects/b1f464b3d677429a8239cd1fb6100d49` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:38.156865 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468141a-7ca1-4f50-b32c-34f88f0a0cb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1f464b3d677429a8239cd1fb6100d49)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:38.159609 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9468141a-7ca1-4f50-b32c-34f88f0a0cb7 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:38.239174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9468141a-7ca1-4f50-b32c-34f88f0a0cb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1f464b3d677429a8239cd1fb6100d49', 'request_id': 'req-9468141a-7ca1-4f50-b32c-34f88f0a0cb7'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:38.246505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9468141a-7ca1-4f50-b32c-34f88f0a0cb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1f464b3d677429a8239cd1fb6100d49', 'request_id': 'req-9468141a-7ca1-4f50-b32c-34f88f0a0cb7'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:38.252635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9468141a-7ca1-4f50-b32c-34f88f0a0cb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1f464b3d677429a8239cd1fb6100d49', 'request_id': 'req-9468141a-7ca1-4f50-b32c-34f88f0a0cb7'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:38.271527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9468141a-7ca1-4f50-b32c-34f88f0a0cb7 admin admin] The token cache is being invalidate because project b1f464b3d677429a8239cd1fb6100d49 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:01:38.271736 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9468141a-7ca1-4f50-b32c-34f88f0a0cb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:38.272662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9468141a-7ca1-4f50-b32c-34f88f0a0cb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:38.275000 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 923/1828] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:01:38 2022] DELETE /identity/v3/projects/b1f464b3d677429a8239cd1fb6100d49 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:38.278980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf2915-9819-4a80-9ff9-bc9611b4c5ff service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:38.289376 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf2915-9819-4a80-9ff9-bc9611b4c5ff service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:38.349956 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f793fd-697d-4bae-8c5d-cb10adc1cb5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:38.350959 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f793fd-697d-4bae-8c5d-cb10adc1cb5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:38.351311 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f793fd-697d-4bae-8c5d-cb10adc1cb5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:38.351650 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f793fd-697d-4bae-8c5d-cb10adc1cb5b admin admin] PATH_INFO: `/v3/users/f4910fe8b73c4d5589dfe96c880a0c9e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:38.352858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f793fd-697d-4bae-8c5d-cb10adc1cb5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4910fe8b73c4d5589dfe96c880a0c9e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:38.354814 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f793fd-697d-4bae-8c5d-cb10adc1cb5b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:38.397760 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:38.414388 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-f9f793fd-697d-4bae-8c5d-cb10adc1cb5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4910fe8b73c4d5589dfe96c880a0c9e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:38.415074 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-f9f793fd-697d-4bae-8c5d-cb10adc1cb5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4910fe8b73c4d5589dfe96c880a0c9e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:38.424896 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-f9f793fd-697d-4bae-8c5d-cb10adc1cb5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4910fe8b73c4d5589dfe96c880a0c9e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:38.425658 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-f9f793fd-697d-4bae-8c5d-cb10adc1cb5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4910fe8b73c4d5589dfe96c880a0c9e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:38.441515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-f9f793fd-697d-4bae-8c5d-cb10adc1cb5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4910fe8b73c4d5589dfe96c880a0c9e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:38.461494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 906/1829] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:01:38 2022] DELETE /identity/v3/users/f4910fe8b73c4d5589dfe96c880a0c9e => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:01:38.925243 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468141a-7ca1-4f50-b32c-34f88f0a0cb7 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:38.927274 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-9468141a-7ca1-4f50-b32c-34f88f0a0cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:38.944109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9468141a-7ca1-4f50-b32c-34f88f0a0cb7 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:39.014747 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54783355-85cc-40ca-9a03-27281a649958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:39.015663 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54783355-85cc-40ca-9a03-27281a649958 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:39.015994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54783355-85cc-40ca-9a03-27281a649958 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:39.016242 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54783355-85cc-40ca-9a03-27281a649958 admin admin] PATH_INFO: `/v3/projects/9667137f74dd4b1ab149c1dabc24ec22` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:39.017445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54783355-85cc-40ca-9a03-27281a649958 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9667137f74dd4b1ab149c1dabc24ec22)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:39.019475 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54783355-85cc-40ca-9a03-27281a649958 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:39.021349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:39.095734 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-54783355-85cc-40ca-9a03-27281a649958 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9667137f74dd4b1ab149c1dabc24ec22', 'request_id': 'req-54783355-85cc-40ca-9a03-27281a649958'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:39.102654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-54783355-85cc-40ca-9a03-27281a649958 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9667137f74dd4b1ab149c1dabc24ec22', 'request_id': 'req-54783355-85cc-40ca-9a03-27281a649958'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:39.109688 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-54783355-85cc-40ca-9a03-27281a649958 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9667137f74dd4b1ab149c1dabc24ec22', 'request_id': 'req-54783355-85cc-40ca-9a03-27281a649958'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:39.122941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-54783355-85cc-40ca-9a03-27281a649958 admin admin] The token cache is being invalidate because project 9667137f74dd4b1ab149c1dabc24ec22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:01:39.123228 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-54783355-85cc-40ca-9a03-27281a649958 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:39.123863 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-54783355-85cc-40ca-9a03-27281a649958 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:39.129060 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 924/1830] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:01:38 2022] DELETE /identity/v3/projects/9667137f74dd4b1ab149c1dabc24ec22 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:39.131258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbc509-7b3c-46cc-ae07-3caccd961192 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:39.131558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbc509-7b3c-46cc-ae07-3caccd961192 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:39.131930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbbc509-7b3c-46cc-ae07-3caccd961192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:39.181719 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-adbbc509-7b3c-46cc-ae07-3caccd961192 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:39.183049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-adbbc509-7b3c-46cc-ae07-3caccd961192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:39.279021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 907/1831] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:01:39.287585 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca21bf-6820-434d-a2ab-86e481d21723 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:39.287879 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca21bf-6820-434d-a2ab-86e481d21723 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:39.288187 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ca21bf-6820-434d-a2ab-86e481d21723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:39.440300 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-e9ca21bf-6820-434d-a2ab-86e481d21723 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:39.441607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e9ca21bf-6820-434d-a2ab-86e481d21723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:39.449896 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 925/1832] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:39.454498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbc509-7b3c-46cc-ae07-3caccd961192 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:39.464438 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbbc509-7b3c-46cc-ae07-3caccd961192 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:39.469581 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefd033-27e3-48f4-a9a1-f218ee2cd0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:39.470472 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefd033-27e3-48f4-a9a1-f218ee2cd0e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:39.470794 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefd033-27e3-48f4-a9a1-f218ee2cd0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:39.471085 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eefd033-27e3-48f4-a9a1-f218ee2cd0e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:39.471807 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefd033-27e3-48f4-a9a1-f218ee2cd0e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:01:39.472309 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefd033-27e3-48f4-a9a1-f218ee2cd0e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:39.473964 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eefd033-27e3-48f4-a9a1-f218ee2cd0e4 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:39.486870 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 908/1833] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:01:39.491537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca21bf-6820-434d-a2ab-86e481d21723 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:39.501397 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ca21bf-6820-434d-a2ab-86e481d21723 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:39.506185 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e989a85-eb20-499b-a398-d6803fc2391f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:39.506913 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e989a85-eb20-499b-a398-d6803fc2391f admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:39.507202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e989a85-eb20-499b-a398-d6803fc2391f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:39.507477 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e989a85-eb20-499b-a398-d6803fc2391f admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:39.508171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e989a85-eb20-499b-a398-d6803fc2391f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:39.510066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e989a85-eb20-499b-a398-d6803fc2391f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:39.565120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 926/1834] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:01:39 2022] POST /identity/v3/projects => generated 373 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:39.569957 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefd033-27e3-48f4-a9a1-f218ee2cd0e4 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:39.581540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eefd033-27e3-48f4-a9a1-f218ee2cd0e4 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:39.650992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c12267-9b72-43ee-89aa-c86800164000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:39.652587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c12267-9b72-43ee-89aa-c86800164000 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:39.653180 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c12267-9b72-43ee-89aa-c86800164000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:39.653643 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c12267-9b72-43ee-89aa-c86800164000 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:39.654519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c12267-9b72-43ee-89aa-c86800164000 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:39.659806 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c12267-9b72-43ee-89aa-c86800164000 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:39.691997 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 909/1835] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:01:39 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:39.694004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e989a85-eb20-499b-a398-d6803fc2391f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:39.713497 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e989a85-eb20-499b-a398-d6803fc2391f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:39.728258 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35070163-6859-444f-b964-83534eae3603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:39.729269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35070163-6859-444f-b964-83534eae3603 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:39.729663 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35070163-6859-444f-b964-83534eae3603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:39.730299 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35070163-6859-444f-b964-83534eae3603 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:39.732252 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35070163-6859-444f-b964-83534eae3603 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:39.736643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35070163-6859-444f-b964-83534eae3603 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:39.757007 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 927/1836] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:01:39.761366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c12267-9b72-43ee-89aa-c86800164000 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:39.778564 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c12267-9b72-43ee-89aa-c86800164000 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:39.784499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83f152c-398b-41f2-a2f8-23f20119d2fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:39.785408 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f152c-398b-41f2-a2f8-23f20119d2fc admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:39.785974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f152c-398b-41f2-a2f8-23f20119d2fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:39.786400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83f152c-398b-41f2-a2f8-23f20119d2fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:39.787419 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f152c-398b-41f2-a2f8-23f20119d2fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:39.789420 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83f152c-398b-41f2-a2f8-23f20119d2fc admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:39.803877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 910/1837] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:01:39.808060 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35070163-6859-444f-b964-83534eae3603 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:39.820073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35070163-6859-444f-b964-83534eae3603 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:39.828383 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa000b-5d5f-4a37-b2de-79960aee0985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:39.829471 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa000b-5d5f-4a37-b2de-79960aee0985 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:39.829898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa000b-5d5f-4a37-b2de-79960aee0985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:39.830319 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faa000b-5d5f-4a37-b2de-79960aee0985 admin admin] PATH_INFO: `/v3/projects/5a1cfcacb5ed448cb73ad5054ed0964b/users/ddc4a97488674b52a9b68a408ab2a5a9/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:39.856460 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa000b-5d5f-4a37-b2de-79960aee0985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a1cfcacb5ed448cb73ad5054ed0964b, user_id=ddc4a97488674b52a9b68a408ab2a5a9, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:39.861503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faa000b-5d5f-4a37-b2de-79960aee0985 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:39.883179 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 928/1838] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:39 2022] PUT /identity/v3/projects/5a1cfcacb5ed448cb73ad5054ed0964b/users/ddc4a97488674b52a9b68a408ab2a5a9/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:39.889166 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fae43-055b-4048-8977-4fe4cea9ec48 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:39.889498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fae43-055b-4048-8977-4fe4cea9ec48 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:39.889784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4fae43-055b-4048-8977-4fe4cea9ec48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:39.910919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-4b4fae43-055b-4048-8977-4fe4cea9ec48 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:39.912235 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-4b4fae43-055b-4048-8977-4fe4cea9ec48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:39.925126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:39.996655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 911/1839] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:40.006680 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa000b-5d5f-4a37-b2de-79960aee0985 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:40.008450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-6faa000b-5d5f-4a37-b2de-79960aee0985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:40.107388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faa000b-5d5f-4a37-b2de-79960aee0985 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:40.111916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6557d829-5012-49dc-bfa5-91af3210ddc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:40.112766 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557d829-5012-49dc-bfa5-91af3210ddc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:40.113457 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557d829-5012-49dc-bfa5-91af3210ddc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:40.113939 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6557d829-5012-49dc-bfa5-91af3210ddc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:40.130564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6557d829-5012-49dc-bfa5-91af3210ddc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:40.132549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6557d829-5012-49dc-bfa5-91af3210ddc2 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:40.150180 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 929/1840] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:41.127828 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fae43-055b-4048-8977-4fe4cea9ec48 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:41.131569 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-4b4fae43-055b-4048-8977-4fe4cea9ec48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:41.166299 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4fae43-055b-4048-8977-4fe4cea9ec48 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:41.173685 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afae845-d31d-468f-9680-2fddca93c42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:41.174791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afae845-d31d-468f-9680-2fddca93c42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:41.175081 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afae845-d31d-468f-9680-2fddca93c42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:41.175357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afae845-d31d-468f-9680-2fddca93c42a admin admin] PATH_INFO: `/v3/users/58ab5db75b234a1fbcc0a5a92973818b` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:41.176977 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afae845-d31d-468f-9680-2fddca93c42a admin admin] RBAC: Authorizing `identity:delete_user(user_id=58ab5db75b234a1fbcc0a5a92973818b)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:41.179817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6afae845-d31d-468f-9680-2fddca93c42a admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:41.256648 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6afae845-d31d-468f-9680-2fddca93c42a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58ab5db75b234a1fbcc0a5a92973818b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:41.257629 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6afae845-d31d-468f-9680-2fddca93c42a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58ab5db75b234a1fbcc0a5a92973818b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:41.274763 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6afae845-d31d-468f-9680-2fddca93c42a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58ab5db75b234a1fbcc0a5a92973818b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:41.275524 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6afae845-d31d-468f-9680-2fddca93c42a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58ab5db75b234a1fbcc0a5a92973818b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:41.298348 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6afae845-d31d-468f-9680-2fddca93c42a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58ab5db75b234a1fbcc0a5a92973818b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:41.333154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 912/1841] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:01:41 2022] DELETE /identity/v3/users/58ab5db75b234a1fbcc0a5a92973818b => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:41.337874 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b44e9-47fe-4a8b-8528-ff6c11d4dbb7 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:41.338337 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b44e9-47fe-4a8b-8528-ff6c11d4dbb7 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:41.338424 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b44e9-47fe-4a8b-8528-ff6c11d4dbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:41.361642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-3e7b44e9-47fe-4a8b-8528-ff6c11d4dbb7 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:41.364187 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-3e7b44e9-47fe-4a8b-8528-ff6c11d4dbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:41.405617 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:41.507045 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 930/1842] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:41.523337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afae845-d31d-468f-9680-2fddca93c42a admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:41.527249 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6afae845-d31d-468f-9680-2fddca93c42a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:41.635543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afae845-d31d-468f-9680-2fddca93c42a admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:41.640785 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20427d67-8ef7-4197-9888-54992ae5f760 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:41.641869 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20427d67-8ef7-4197-9888-54992ae5f760 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:41.642365 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20427d67-8ef7-4197-9888-54992ae5f760 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:41.642889 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20427d67-8ef7-4197-9888-54992ae5f760 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:41.654634 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20427d67-8ef7-4197-9888-54992ae5f760 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:41.657426 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20427d67-8ef7-4197-9888-54992ae5f760 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:41.678650 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 913/1843] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:41.975754 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0b061-c976-4911-a48d-a9ccb560d21c None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:41.976053 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0b061-c976-4911-a48d-a9ccb560d21c None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:41.976337 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce0b061-c976-4911-a48d-a9ccb560d21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:42.034757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-4ce0b061-c976-4911-a48d-a9ccb560d21c None None] MFA Rules not processed for user `ddc4a97488674b52a9b68a408ab2a5a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:42.036297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-4ce0b061-c976-4911-a48d-a9ccb560d21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:42.210472 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 931/1844] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:41 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:42.215519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20427d67-8ef7-4197-9888-54992ae5f760 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:42.217076 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-20427d67-8ef7-4197-9888-54992ae5f760 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:42.242031 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20427d67-8ef7-4197-9888-54992ae5f760 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:42.248484 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d3a42-ab62-4c7b-88cc-376f8805e330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:42.249979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d3a42-ab62-4c7b-88cc-376f8805e330 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:42.250395 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d3a42-ab62-4c7b-88cc-376f8805e330 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:42.250802 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05d3a42-ab62-4c7b-88cc-376f8805e330 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:42.251606 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d3a42-ab62-4c7b-88cc-376f8805e330 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:42.255662 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05d3a42-ab62-4c7b-88cc-376f8805e330 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:42.327440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 914/1845] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:01:42 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:42.327676 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0b061-c976-4911-a48d-a9ccb560d21c None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:42.345589 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce0b061-c976-4911-a48d-a9ccb560d21c None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:42.532645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464014f-735a-4b4f-887f-0249f8b05a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:42.533687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464014f-735a-4b4f-887f-0249f8b05a88 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:42.533979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464014f-735a-4b4f-887f-0249f8b05a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:42.534258 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6464014f-735a-4b4f-887f-0249f8b05a88 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:42.535000 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464014f-735a-4b4f-887f-0249f8b05a88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:42.537693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6464014f-735a-4b4f-887f-0249f8b05a88 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:42.576183 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 932/1846] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:01:42 2022] POST /identity/v3/users => generated 292 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:01:42.579949 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d3a42-ab62-4c7b-88cc-376f8805e330 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:42.590668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05d3a42-ab62-4c7b-88cc-376f8805e330 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:42.596969 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b7204-57d2-42c8-903b-ca39f3281148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:42.598057 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b7204-57d2-42c8-903b-ca39f3281148 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:42.598172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b7204-57d2-42c8-903b-ca39f3281148 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:42.598473 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4b7204-57d2-42c8-903b-ca39f3281148 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:42.599998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b7204-57d2-42c8-903b-ca39f3281148 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:42.601805 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4b7204-57d2-42c8-903b-ca39f3281148 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:42.605999 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:42.623832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464014f-735a-4b4f-887f-0249f8b05a88 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:42.623970 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 915/1847] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:42.647068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6464014f-735a-4b4f-887f-0249f8b05a88 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:42.666649 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4edf5-2b41-4a5a-9341-0434acea0d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:42.668391 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4edf5-2b41-4a5a-9341-0434acea0d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:42.668957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4edf5-2b41-4a5a-9341-0434acea0d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:42.670465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea4edf5-2b41-4a5a-9341-0434acea0d48 admin admin] PATH_INFO: `/v3/projects/01a566c0abd24253857e5926c1f653b5/users/544705b920b141fcacfe9fa0337fbae4/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:42.706532 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4edf5-2b41-4a5a-9341-0434acea0d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01a566c0abd24253857e5926c1f653b5, user_id=544705b920b141fcacfe9fa0337fbae4, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:42.713013 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea4edf5-2b41-4a5a-9341-0434acea0d48 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:42.748163 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 933/1848] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:42 2022] PUT /identity/v3/projects/01a566c0abd24253857e5926c1f653b5/users/544705b920b141fcacfe9fa0337fbae4/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:01:42.748895 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b7204-57d2-42c8-903b-ca39f3281148 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:42.761871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4b7204-57d2-42c8-903b-ca39f3281148 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:42.879790 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56912f-d32a-4cae-92ee-aebcc8d1efda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:42.880500 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56912f-d32a-4cae-92ee-aebcc8d1efda admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:42.880789 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56912f-d32a-4cae-92ee-aebcc8d1efda admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:42.881069 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56912f-d32a-4cae-92ee-aebcc8d1efda admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:42.885228 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56912f-d32a-4cae-92ee-aebcc8d1efda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:42.892453 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db56912f-d32a-4cae-92ee-aebcc8d1efda admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:42.934670 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4edf5-2b41-4a5a-9341-0434acea0d48 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:42.954331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 916/1849] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:42 2022] GET /identity/v3/roles => generated 1420 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:42.955847 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea4edf5-2b41-4a5a-9341-0434acea0d48 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:42.964771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b43979-0e23-4b44-a067-94bafbe73ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:42.968808 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b43979-0e23-4b44-a067-94bafbe73ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:42.969110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b43979-0e23-4b44-a067-94bafbe73ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:42.969385 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b43979-0e23-4b44-a067-94bafbe73ba3 admin admin] PATH_INFO: `/v3/projects/01a566c0abd24253857e5926c1f653b5/users/544705b920b141fcacfe9fa0337fbae4/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:42.974899 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b43979-0e23-4b44-a067-94bafbe73ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01a566c0abd24253857e5926c1f653b5, user_id=544705b920b141fcacfe9fa0337fbae4, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:42.983089 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b43979-0e23-4b44-a067-94bafbe73ba3 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:43.009496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 934/1850] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:42 2022] PUT /identity/v3/projects/01a566c0abd24253857e5926c1f653b5/users/544705b920b141fcacfe9fa0337fbae4/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:43.040299 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56912f-d32a-4cae-92ee-aebcc8d1efda admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:43.043402 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-db56912f-d32a-4cae-92ee-aebcc8d1efda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:43.074432 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56912f-d32a-4cae-92ee-aebcc8d1efda admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:43.254554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb3a4640-d298-4d91-9b38-bb2ae38494aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:43.256890 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a4640-d298-4d91-9b38-bb2ae38494aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:43.259271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a4640-d298-4d91-9b38-bb2ae38494aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:43.260035 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3a4640-d298-4d91-9b38-bb2ae38494aa admin admin] PATH_INFO: `/v3/projects/bd58c1c77685485ca8fd862e03dbbd89` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:43.265029 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a4640-d298-4d91-9b38-bb2ae38494aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd58c1c77685485ca8fd862e03dbbd89)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:43.267728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb3a4640-d298-4d91-9b38-bb2ae38494aa admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:43.385296 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cb3a4640-d298-4d91-9b38-bb2ae38494aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd58c1c77685485ca8fd862e03dbbd89', 'request_id': 'req-cb3a4640-d298-4d91-9b38-bb2ae38494aa'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:43.398674 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cb3a4640-d298-4d91-9b38-bb2ae38494aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd58c1c77685485ca8fd862e03dbbd89', 'request_id': 'req-cb3a4640-d298-4d91-9b38-bb2ae38494aa'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:43.415418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cb3a4640-d298-4d91-9b38-bb2ae38494aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd58c1c77685485ca8fd862e03dbbd89', 'request_id': 'req-cb3a4640-d298-4d91-9b38-bb2ae38494aa'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:43.449462 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cb3a4640-d298-4d91-9b38-bb2ae38494aa admin admin] The token cache is being invalidate because project bd58c1c77685485ca8fd862e03dbbd89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:01:43.449755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cb3a4640-d298-4d91-9b38-bb2ae38494aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:43.450443 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cb3a4640-d298-4d91-9b38-bb2ae38494aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:43.453315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 917/1851] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:01:43 2022] DELETE /identity/v3/projects/bd58c1c77685485ca8fd862e03dbbd89 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:43.457084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b43979-0e23-4b44-a067-94bafbe73ba3 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:43.459525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-81b43979-0e23-4b44-a067-94bafbe73ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:43.485261 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b43979-0e23-4b44-a067-94bafbe73ba3 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:43.505958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:43.593444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866eeea1-0e81-4031-9930-b51066fe8a47 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:43.593994 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866eeea1-0e81-4031-9930-b51066fe8a47 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:43.594528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866eeea1-0e81-4031-9930-b51066fe8a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:43.633757 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-866eeea1-0e81-4031-9930-b51066fe8a47 None None] MFA Rules not processed for user `01dd0e5661f44d05a01d7a5e82dbadf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:43.637337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-866eeea1-0e81-4031-9930-b51066fe8a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:43.680612 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:43.696453 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8950c69-dadd-448e-9431-a2edb727ec39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:43.698261 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8950c69-dadd-448e-9431-a2edb727ec39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:43.698559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8950c69-dadd-448e-9431-a2edb727ec39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:43.699095 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8950c69-dadd-448e-9431-a2edb727ec39 admin admin] PATH_INFO: `/v3/users/1e320d9f3ab046718251c691efd8d36b` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:43.700533 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8950c69-dadd-448e-9431-a2edb727ec39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e320d9f3ab046718251c691efd8d36b)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:43.702764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8950c69-dadd-448e-9431-a2edb727ec39 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:43.804813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e8950c69-dadd-448e-9431-a2edb727ec39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e320d9f3ab046718251c691efd8d36b'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:43.822935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e8950c69-dadd-448e-9431-a2edb727ec39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e320d9f3ab046718251c691efd8d36b'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:43.826301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e8950c69-dadd-448e-9431-a2edb727ec39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e320d9f3ab046718251c691efd8d36b'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:43.852979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e8950c69-dadd-448e-9431-a2edb727ec39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e320d9f3ab046718251c691efd8d36b'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:43.853677 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e8950c69-dadd-448e-9431-a2edb727ec39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e320d9f3ab046718251c691efd8d36b'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:43.882548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 918/1852] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:43.883838 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866eeea1-0e81-4031-9930-b51066fe8a47 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:43.887348 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-866eeea1-0e81-4031-9930-b51066fe8a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:43.892734 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 935/1853] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:01:43 2022] DELETE /identity/v3/users/1e320d9f3ab046718251c691efd8d36b => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:43.924180 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866eeea1-0e81-4031-9930-b51066fe8a47 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:44.076252 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87037b-6b75-4dca-b8cd-3f1ba28bb864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:44.077148 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87037b-6b75-4dca-b8cd-3f1ba28bb864 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:44.078082 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87037b-6b75-4dca-b8cd-3f1ba28bb864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:44.078408 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87037b-6b75-4dca-b8cd-3f1ba28bb864 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:44.079271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87037b-6b75-4dca-b8cd-3f1ba28bb864 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:44.081482 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a87037b-6b75-4dca-b8cd-3f1ba28bb864 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:44.144533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 919/1854] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:01:43 2022] POST /identity/v3/projects => generated 373 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:44.146319 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8950c69-dadd-448e-9431-a2edb727ec39 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:44.148143 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e8950c69-dadd-448e-9431-a2edb727ec39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:44.171929 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8950c69-dadd-448e-9431-a2edb727ec39 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:44.353506 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41486589-da7a-4996-a0a5-ad569a48f12f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:44.354264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41486589-da7a-4996-a0a5-ad569a48f12f admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:44.354852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41486589-da7a-4996-a0a5-ad569a48f12f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:44.355136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41486589-da7a-4996-a0a5-ad569a48f12f admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:44.355853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41486589-da7a-4996-a0a5-ad569a48f12f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:44.357792 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41486589-da7a-4996-a0a5-ad569a48f12f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:44.394156 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 936/1855] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:01:44 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:01:44.396750 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87037b-6b75-4dca-b8cd-3f1ba28bb864 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:44.415982 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87037b-6b75-4dca-b8cd-3f1ba28bb864 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:44.429224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1b43e-4e0e-48e3-98a9-761acbac67b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:44.429450 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1b43e-4e0e-48e3-98a9-761acbac67b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:44.430505 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1b43e-4e0e-48e3-98a9-761acbac67b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:44.431094 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f1b43e-4e0e-48e3-98a9-761acbac67b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:44.432475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1b43e-4e0e-48e3-98a9-761acbac67b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:44.435391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f1b43e-4e0e-48e3-98a9-761acbac67b7 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:44.453417 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 920/1856] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:01:44.458360 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41486589-da7a-4996-a0a5-ad569a48f12f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:44.471693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41486589-da7a-4996-a0a5-ad569a48f12f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:44.481660 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dfb258-2501-4bcf-84d7-b91d76a9adad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:44.482886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfb258-2501-4bcf-84d7-b91d76a9adad admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:44.483747 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfb258-2501-4bcf-84d7-b91d76a9adad admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:44.484161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dfb258-2501-4bcf-84d7-b91d76a9adad admin admin] PATH_INFO: `/v3/projects/1c9163c4876f4500883459a1afca5edd/users/105edf55cdfc4085bde1ccfffddd325e/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:44.509921 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dfb258-2501-4bcf-84d7-b91d76a9adad admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c9163c4876f4500883459a1afca5edd, user_id=105edf55cdfc4085bde1ccfffddd325e, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:44.513265 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dfb258-2501-4bcf-84d7-b91d76a9adad admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:44.518379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:44.542380 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 937/1857] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:44 2022] PUT /identity/v3/projects/1c9163c4876f4500883459a1afca5edd/users/105edf55cdfc4085bde1ccfffddd325e/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:44.543758 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1b43e-4e0e-48e3-98a9-761acbac67b7 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:44.564318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f1b43e-4e0e-48e3-98a9-761acbac67b7 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:44.665823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792bdd6-3ed9-4d43-aab8-a62a8ae6937a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:44.666559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792bdd6-3ed9-4d43-aab8-a62a8ae6937a admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:44.666922 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792bdd6-3ed9-4d43-aab8-a62a8ae6937a admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:44.667202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7792bdd6-3ed9-4d43-aab8-a62a8ae6937a admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:44.667934 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792bdd6-3ed9-4d43-aab8-a62a8ae6937a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:44.669736 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7792bdd6-3ed9-4d43-aab8-a62a8ae6937a admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:44.689895 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 921/1858] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:44.691291 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dfb258-2501-4bcf-84d7-b91d76a9adad admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:44.704246 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dfb258-2501-4bcf-84d7-b91d76a9adad admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:44.709720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf830112-621e-4c80-84fc-a5ff0bd9ab76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:44.710936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf830112-621e-4c80-84fc-a5ff0bd9ab76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:44.711320 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf830112-621e-4c80-84fc-a5ff0bd9ab76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:44.711695 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf830112-621e-4c80-84fc-a5ff0bd9ab76 admin admin] PATH_INFO: `/v3/projects/1c9163c4876f4500883459a1afca5edd/users/105edf55cdfc4085bde1ccfffddd325e/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:44.714335 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf830112-621e-4c80-84fc-a5ff0bd9ab76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c9163c4876f4500883459a1afca5edd, user_id=105edf55cdfc4085bde1ccfffddd325e, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:44.717439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf830112-621e-4c80-84fc-a5ff0bd9ab76 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:44.735849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 938/1859] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:44 2022] PUT /identity/v3/projects/1c9163c4876f4500883459a1afca5edd/users/105edf55cdfc4085bde1ccfffddd325e/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:44.889411 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792bdd6-3ed9-4d43-aab8-a62a8ae6937a admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:44.892332 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-7792bdd6-3ed9-4d43-aab8-a62a8ae6937a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:44.915478 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7792bdd6-3ed9-4d43-aab8-a62a8ae6937a admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:45.008157 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:45.087292 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668d92b7-694e-4234-b576-55a29c607fe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:45.088397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d92b7-694e-4234-b576-55a29c607fe0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:45.089513 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d92b7-694e-4234-b576-55a29c607fe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:45.089983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668d92b7-694e-4234-b576-55a29c607fe0 admin admin] PATH_INFO: `/v3/projects/b1a49045ea794afa8d7c697052559550` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:45.091493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d92b7-694e-4234-b576-55a29c607fe0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1a49045ea794afa8d7c697052559550)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:45.093871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668d92b7-694e-4234-b576-55a29c607fe0 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:45.098908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff19732-66ec-45ea-bab8-6e58f333c0b5 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:45.102542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff19732-66ec-45ea-bab8-6e58f333c0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:45.103067 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff19732-66ec-45ea-bab8-6e58f333c0b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:45.218304 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-668d92b7-694e-4234-b576-55a29c607fe0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1a49045ea794afa8d7c697052559550', 'request_id': 'req-668d92b7-694e-4234-b576-55a29c607fe0'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:45.230583 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-fff19732-66ec-45ea-bab8-6e58f333c0b5 None None] MFA Rules not processed for user `544705b920b141fcacfe9fa0337fbae4`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:45.232272 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-668d92b7-694e-4234-b576-55a29c607fe0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1a49045ea794afa8d7c697052559550', 'request_id': 'req-668d92b7-694e-4234-b576-55a29c607fe0'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:45.242627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-fff19732-66ec-45ea-bab8-6e58f333c0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:45.256975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-668d92b7-694e-4234-b576-55a29c607fe0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1a49045ea794afa8d7c697052559550', 'request_id': 'req-668d92b7-694e-4234-b576-55a29c607fe0'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:45.289911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-668d92b7-694e-4234-b576-55a29c607fe0 admin admin] The token cache is being invalidate because project b1a49045ea794afa8d7c697052559550 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:01:45.290213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-668d92b7-694e-4234-b576-55a29c607fe0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:45.292511 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-668d92b7-694e-4234-b576-55a29c607fe0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:45.294754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 922/1860] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:01:44 2022] DELETE /identity/v3/projects/b1a49045ea794afa8d7c697052559550 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:45.301198 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceefba3-759e-4545-b1e3-e35c7c16fa7d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:45.301722 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceefba3-759e-4545-b1e3-e35c7c16fa7d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:45.302364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceefba3-759e-4545-b1e3-e35c7c16fa7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:45.359522 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-aceefba3-759e-4545-b1e3-e35c7c16fa7d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:45.364354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-aceefba3-759e-4545-b1e3-e35c7c16fa7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:45.471502 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 939/1861] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:45 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:45.481291 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff19732-66ec-45ea-bab8-6e58f333c0b5 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:45.482795 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-fff19732-66ec-45ea-bab8-6e58f333c0b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:45.495325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 923/1862] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:01:45.497384 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06125b-85e9-405c-bb75-444fb814ce83 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:45.497687 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06125b-85e9-405c-bb75-444fb814ce83 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:45.498187 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a06125b-85e9-405c-bb75-444fb814ce83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:45.551648 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-5a06125b-85e9-405c-bb75-444fb814ce83 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:45.555859 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-5a06125b-85e9-405c-bb75-444fb814ce83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:45.588974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 924/1863] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:01:45.591920 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06125b-85e9-405c-bb75-444fb814ce83 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:45.611528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff19732-66ec-45ea-bab8-6e58f333c0b5 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:45.619108 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a06125b-85e9-405c-bb75-444fb814ce83 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:45.622544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068748a0-ebb5-409b-8a2a-62f95139fd1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:45.623503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068748a0-ebb5-409b-8a2a-62f95139fd1e service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:45.623864 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068748a0-ebb5-409b-8a2a-62f95139fd1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:45.624145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068748a0-ebb5-409b-8a2a-62f95139fd1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:45.627272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-068748a0-ebb5-409b-8a2a-62f95139fd1e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:45.632410 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566222e-4ab6-4a72-9663-c1d0569cad80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:45.633608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566222e-4ab6-4a72-9663-c1d0569cad80 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:45.633913 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566222e-4ab6-4a72-9663-c1d0569cad80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:45.634200 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566222e-4ab6-4a72-9663-c1d0569cad80 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:45.634982 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566222e-4ab6-4a72-9663-c1d0569cad80 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:01:45.635302 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566222e-4ab6-4a72-9663-c1d0569cad80 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:45.644902 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566222e-4ab6-4a72-9663-c1d0569cad80 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:45.694295 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 925/1864] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:01:45.695426 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566222e-4ab6-4a72-9663-c1d0569cad80 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:45.721557 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566222e-4ab6-4a72-9663-c1d0569cad80 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:45.746038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467bf597-cd74-4464-b7a4-6694cd8367d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:45.747264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bf597-cd74-4464-b7a4-6694cd8367d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:45.747561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bf597-cd74-4464-b7a4-6694cd8367d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:45.748001 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467bf597-cd74-4464-b7a4-6694cd8367d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:45.748719 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467bf597-cd74-4464-b7a4-6694cd8367d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:45.750143 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068748a0-ebb5-409b-8a2a-62f95139fd1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:45.751618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068748a0-ebb5-409b-8a2a-62f95139fd1e service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:45.751874 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467bf597-cd74-4464-b7a4-6694cd8367d0 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:45.774740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 940/1865] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:45.799312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 926/1866] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:01:45 2022] POST /identity/v3/projects => generated 369 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:01:45.800512 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068748a0-ebb5-409b-8a2a-62f95139fd1e service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:45.814216 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068748a0-ebb5-409b-8a2a-62f95139fd1e service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:45.848137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:45.937003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2109254-0522-4c80-8737-6fea98d003f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:45.937702 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2109254-0522-4c80-8737-6fea98d003f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:45.937992 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2109254-0522-4c80-8737-6fea98d003f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:45.938883 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2109254-0522-4c80-8737-6fea98d003f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:45.939019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2109254-0522-4c80-8737-6fea98d003f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:45.940924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2109254-0522-4c80-8737-6fea98d003f6 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:45.978158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 941/1867] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:01:45 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:45.985378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467bf597-cd74-4464-b7a4-6694cd8367d0 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:46.004415 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467bf597-cd74-4464-b7a4-6694cd8367d0 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:46.011835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298394f-dbd6-4530-ad78-7b83fc946906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:46.013303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298394f-dbd6-4530-ad78-7b83fc946906 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:46.013860 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298394f-dbd6-4530-ad78-7b83fc946906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:46.014356 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298394f-dbd6-4530-ad78-7b83fc946906 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:46.015939 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298394f-dbd6-4530-ad78-7b83fc946906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:46.019008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298394f-dbd6-4530-ad78-7b83fc946906 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:46.035509 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 927/1868] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:01:46.039221 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2109254-0522-4c80-8737-6fea98d003f6 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:46.055654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2109254-0522-4c80-8737-6fea98d003f6 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:46.069422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e83a98-f33e-4f7c-ad4f-ae74bb2b2a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:46.070183 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e83a98-f33e-4f7c-ad4f-ae74bb2b2a90 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:46.070413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e83a98-f33e-4f7c-ad4f-ae74bb2b2a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:46.070729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e83a98-f33e-4f7c-ad4f-ae74bb2b2a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:46.071480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e83a98-f33e-4f7c-ad4f-ae74bb2b2a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:46.073102 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e83a98-f33e-4f7c-ad4f-ae74bb2b2a90 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:46.100196 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 942/1869] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:46.104986 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298394f-dbd6-4530-ad78-7b83fc946906 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:46.131501 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:46.164624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298394f-dbd6-4530-ad78-7b83fc946906 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:46.173673 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdab84f-100b-4a72-8540-40214ee002fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:46.174823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdab84f-100b-4a72-8540-40214ee002fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:46.177320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdab84f-100b-4a72-8540-40214ee002fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:46.177709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdab84f-100b-4a72-8540-40214ee002fd admin admin] PATH_INFO: `/v3/projects/7be16480e777410aa42ec15347ddda03/users/333ae9419f9c4a8a8710ed169b27acb0/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:46.199193 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdab84f-100b-4a72-8540-40214ee002fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=7be16480e777410aa42ec15347ddda03, user_id=333ae9419f9c4a8a8710ed169b27acb0, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:46.203034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdab84f-100b-4a72-8540-40214ee002fd admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:46.245403 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c469a5b-41d6-4dfd-9bb4-eed9362fe565 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:46.246563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c469a5b-41d6-4dfd-9bb4-eed9362fe565 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:46.246896 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c469a5b-41d6-4dfd-9bb4-eed9362fe565 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:46.249252 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 928/1870] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:46 2022] PUT /identity/v3/projects/7be16480e777410aa42ec15347ddda03/users/333ae9419f9c4a8a8710ed169b27acb0/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:46.296634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-3c469a5b-41d6-4dfd-9bb4-eed9362fe565 None None] MFA Rules not processed for user `333ae9419f9c4a8a8710ed169b27acb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:46.298082 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-3c469a5b-41d6-4dfd-9bb4-eed9362fe565 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:46.445332 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdab84f-100b-4a72-8540-40214ee002fd admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:46.454723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 943/1871] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:46 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:46.546505 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdab84f-100b-4a72-8540-40214ee002fd admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:46.568626 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d4239-48fd-4db7-a1d9-6171b8089f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:46.570041 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d4239-48fd-4db7-a1d9-6171b8089f53 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:46.570338 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d4239-48fd-4db7-a1d9-6171b8089f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:46.571781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624d4239-48fd-4db7-a1d9-6171b8089f53 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:46.573071 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d4239-48fd-4db7-a1d9-6171b8089f53 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:46.575087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624d4239-48fd-4db7-a1d9-6171b8089f53 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:46.619627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c469a5b-41d6-4dfd-9bb4-eed9362fe565 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:46.621947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 929/1872] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:01:46 2022] POST /identity/v3/projects => generated 369 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:46.635476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c469a5b-41d6-4dfd-9bb4-eed9362fe565 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:46.715655 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0c8e9-0398-4fe4-87c8-9d1d101870df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:46.716815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0c8e9-0398-4fe4-87c8-9d1d101870df admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:46.717397 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0c8e9-0398-4fe4-87c8-9d1d101870df admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:46.718368 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf0c8e9-0398-4fe4-87c8-9d1d101870df admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:46.719602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0c8e9-0398-4fe4-87c8-9d1d101870df admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:46.722577 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf0c8e9-0398-4fe4-87c8-9d1d101870df admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:46.760405 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 944/1873] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:01:46 2022] POST /identity/v3/users => generated 291 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:46.764286 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d4239-48fd-4db7-a1d9-6171b8089f53 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:46.775933 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624d4239-48fd-4db7-a1d9-6171b8089f53 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:46.784362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8305fb-6423-4c2c-8058-99781b448184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:46.785254 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8305fb-6423-4c2c-8058-99781b448184 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:46.785613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8305fb-6423-4c2c-8058-99781b448184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:46.785959 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8305fb-6423-4c2c-8058-99781b448184 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:46.787468 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8305fb-6423-4c2c-8058-99781b448184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:46.789549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8305fb-6423-4c2c-8058-99781b448184 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:46.811898 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 930/1874] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:46.817601 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0c8e9-0398-4fe4-87c8-9d1d101870df admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:46.835154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf0c8e9-0398-4fe4-87c8-9d1d101870df admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:46.840389 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be753f-0a4f-4a38-9e03-882b6465c10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:46.841324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be753f-0a4f-4a38-9e03-882b6465c10b admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:46.841608 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be753f-0a4f-4a38-9e03-882b6465c10b admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:46.841891 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06be753f-0a4f-4a38-9e03-882b6465c10b admin admin] PATH_INFO: `/v3/projects/4c6af46c467b49d883798bdbd4ac01c0/users/253bb08ca26f44b78f7e71d84820bd49/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:46.863288 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be753f-0a4f-4a38-9e03-882b6465c10b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6af46c467b49d883798bdbd4ac01c0, user_id=253bb08ca26f44b78f7e71d84820bd49, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:46.866205 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06be753f-0a4f-4a38-9e03-882b6465c10b admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:46.872884 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:46.892679 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 945/1875] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:46 2022] PUT /identity/v3/projects/4c6af46c467b49d883798bdbd4ac01c0/users/253bb08ca26f44b78f7e71d84820bd49/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:46.894050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8305fb-6423-4c2c-8058-99781b448184 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:46.909429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8305fb-6423-4c2c-8058-99781b448184 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:47.035185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72865911-7e34-4c4b-b6ed-d3281c219204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:47.035947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72865911-7e34-4c4b-b6ed-d3281c219204 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:47.036238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72865911-7e34-4c4b-b6ed-d3281c219204 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:47.036630 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72865911-7e34-4c4b-b6ed-d3281c219204 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:47.037557 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72865911-7e34-4c4b-b6ed-d3281c219204 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:47.039286 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72865911-7e34-4c4b-b6ed-d3281c219204 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:47.060338 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 931/1876] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:47.064430 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be753f-0a4f-4a38-9e03-882b6465c10b admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:47.080473 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06be753f-0a4f-4a38-9e03-882b6465c10b admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:47.110540 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552ace0-d82d-400f-a604-49adeaa373de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:47.111582 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552ace0-d82d-400f-a604-49adeaa373de admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:47.111867 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552ace0-d82d-400f-a604-49adeaa373de admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:47.112147 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552ace0-d82d-400f-a604-49adeaa373de admin admin] PATH_INFO: `/v3/projects/4c6af46c467b49d883798bdbd4ac01c0/users/253bb08ca26f44b78f7e71d84820bd49/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:47.117337 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552ace0-d82d-400f-a604-49adeaa373de admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c6af46c467b49d883798bdbd4ac01c0, user_id=253bb08ca26f44b78f7e71d84820bd49, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:47.121105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552ace0-d82d-400f-a604-49adeaa373de admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:47.164815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 946/1877] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:47 2022] PUT /identity/v3/projects/4c6af46c467b49d883798bdbd4ac01c0/users/253bb08ca26f44b78f7e71d84820bd49/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:47.175568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bf086-77f7-46ec-b6ec-deffbc08eca7 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:47.176222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bf086-77f7-46ec-b6ec-deffbc08eca7 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:47.176650 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13bf086-77f7-46ec-b6ec-deffbc08eca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:47.301426 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-b13bf086-77f7-46ec-b6ec-deffbc08eca7 None None] MFA Rules not processed for user `253bb08ca26f44b78f7e71d84820bd49`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:47.302739 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-b13bf086-77f7-46ec-b6ec-deffbc08eca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:47.425610 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:47.508019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 932/1878] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:47 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:47.513982 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552ace0-d82d-400f-a604-49adeaa373de admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:47.516015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7552ace0-d82d-400f-a604-49adeaa373de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:47.573256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bf086-77f7-46ec-b6ec-deffbc08eca7 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:47.715551 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552ace0-d82d-400f-a604-49adeaa373de admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:47.720978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3991502-ef03-4069-bcbb-abce0e64f292 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:47.721717 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3991502-ef03-4069-bcbb-abce0e64f292 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:47.722007 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3991502-ef03-4069-bcbb-abce0e64f292 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:47.722284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3991502-ef03-4069-bcbb-abce0e64f292 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:47.731947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13bf086-77f7-46ec-b6ec-deffbc08eca7 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:47.744169 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b39d40-71fa-4079-b571-46581d80fbfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:47.745615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b39d40-71fa-4079-b571-46581d80fbfe service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:47.745929 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b39d40-71fa-4079-b571-46581d80fbfe service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:47.746505 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b39d40-71fa-4079-b571-46581d80fbfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:47.750300 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3991502-ef03-4069-bcbb-abce0e64f292 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:47.751805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3991502-ef03-4069-bcbb-abce0e64f292 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:47.774778 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 947/1879] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:47 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:47.899678 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3991502-ef03-4069-bcbb-abce0e64f292 service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:47.913182 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3991502-ef03-4069-bcbb-abce0e64f292 service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:47.923218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32a3dc1-0ca3-4704-afc5-81eb8ba9d326 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:47.924186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a3dc1-0ca3-4704-afc5-81eb8ba9d326 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:47.924595 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a3dc1-0ca3-4704-afc5-81eb8ba9d326 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:47.925048 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32a3dc1-0ca3-4704-afc5-81eb8ba9d326 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:47.971008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b39d40-71fa-4079-b571-46581d80fbfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:47.973220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b39d40-71fa-4079-b571-46581d80fbfe service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:47.991447 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 933/1880] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:47 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:01:48.090432 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a3dc1-0ca3-4704-afc5-81eb8ba9d326 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:48.094447 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32a3dc1-0ca3-4704-afc5-81eb8ba9d326 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:48.121215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 948/1881] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:47 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:52.249440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b39d40-71fa-4079-b571-46581d80fbfe service cinder] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:52.348122 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b39d40-71fa-4079-b571-46581d80fbfe service cinder] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:52.355022 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4c06a-6ecd-40da-883d-24ba6002974d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:52.356549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4c06a-6ecd-40da-883d-24ba6002974d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:52.357205 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4c06a-6ecd-40da-883d-24ba6002974d admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:52.357750 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e4c06a-6ecd-40da-883d-24ba6002974d admin admin] PATH_INFO: `/v3/users/333ae9419f9c4a8a8710ed169b27acb0` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:52.360086 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e4c06a-6ecd-40da-883d-24ba6002974d admin admin] RBAC: Authorizing `identity:delete_user(user_id=333ae9419f9c4a8a8710ed169b27acb0)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:52.362507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e4c06a-6ecd-40da-883d-24ba6002974d admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:52.412620 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:52.450174 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c1e4c06a-6ecd-40da-883d-24ba6002974d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '333ae9419f9c4a8a8710ed169b27acb0'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:52.451669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c1e4c06a-6ecd-40da-883d-24ba6002974d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '333ae9419f9c4a8a8710ed169b27acb0'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:52.463652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c1e4c06a-6ecd-40da-883d-24ba6002974d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '333ae9419f9c4a8a8710ed169b27acb0'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:52.464744 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c1e4c06a-6ecd-40da-883d-24ba6002974d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '333ae9419f9c4a8a8710ed169b27acb0'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:52.508380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c1e4c06a-6ecd-40da-883d-24ba6002974d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '333ae9419f9c4a8a8710ed169b27acb0'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:52.533317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 934/1882] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:01:52 2022] DELETE /identity/v3/users/333ae9419f9c4a8a8710ed169b27acb0 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:52.539098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfc13c-48e1-4847-b6c3-0f5a26172976 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:52.543665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfc13c-48e1-4847-b6c3-0f5a26172976 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:52.543967 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abfc13c-48e1-4847-b6c3-0f5a26172976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:52.629455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-4abfc13c-48e1-4847-b6c3-0f5a26172976 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:52.630924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-4abfc13c-48e1-4847-b6c3-0f5a26172976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:52.823013 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 949/1883] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:52.864580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4c06a-6ecd-40da-883d-24ba6002974d admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:52.868864 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c1e4c06a-6ecd-40da-883d-24ba6002974d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:52.978388 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e4c06a-6ecd-40da-883d-24ba6002974d admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:52.989063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a1978-39f8-443b-b453-ebb876db49b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:52.990143 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a1978-39f8-443b-b453-ebb876db49b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:52.990536 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a1978-39f8-443b-b453-ebb876db49b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:52.990858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a1978-39f8-443b-b453-ebb876db49b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:53.015729 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a1978-39f8-443b-b453-ebb876db49b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:53.019106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a1978-39f8-443b-b453-ebb876db49b4 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:53.043393 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 935/1884] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:54.099531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abfc13c-48e1-4847-b6c3-0f5a26172976 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:54.099991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-4abfc13c-48e1-4847-b6c3-0f5a26172976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:54.121502 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abfc13c-48e1-4847-b6c3-0f5a26172976 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:54.137362 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68880e68-8386-4a39-bcca-acc3caac52d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:54.138549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68880e68-8386-4a39-bcca-acc3caac52d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:54.139057 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68880e68-8386-4a39-bcca-acc3caac52d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:54.139550 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68880e68-8386-4a39-bcca-acc3caac52d8 admin admin] PATH_INFO: `/v3/projects/7be16480e777410aa42ec15347ddda03` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:54.141544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68880e68-8386-4a39-bcca-acc3caac52d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7be16480e777410aa42ec15347ddda03)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:54.144129 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68880e68-8386-4a39-bcca-acc3caac52d8 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:54.192496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:54.254166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-68880e68-8386-4a39-bcca-acc3caac52d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7be16480e777410aa42ec15347ddda03', 'request_id': 'req-68880e68-8386-4a39-bcca-acc3caac52d8'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:54.262983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-68880e68-8386-4a39-bcca-acc3caac52d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7be16480e777410aa42ec15347ddda03', 'request_id': 'req-68880e68-8386-4a39-bcca-acc3caac52d8'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:54.283353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-68880e68-8386-4a39-bcca-acc3caac52d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7be16480e777410aa42ec15347ddda03', 'request_id': 'req-68880e68-8386-4a39-bcca-acc3caac52d8'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:54.303783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-68880e68-8386-4a39-bcca-acc3caac52d8 admin admin] The token cache is being invalidate because project 7be16480e777410aa42ec15347ddda03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:01:54.304084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-68880e68-8386-4a39-bcca-acc3caac52d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:54.304760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-68880e68-8386-4a39-bcca-acc3caac52d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:54.309442 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 950/1885] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:01:54 2022] DELETE /identity/v3/projects/7be16480e777410aa42ec15347ddda03 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:54.313926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a1978-39f8-443b-b453-ebb876db49b4 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:54.329618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a1978-39f8-443b-b453-ebb876db49b4 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:54.515117 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e39ee8-d82a-48a1-af4d-8ff7a9b97ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:54.516091 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e39ee8-d82a-48a1-af4d-8ff7a9b97ec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:54.516314 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e39ee8-d82a-48a1-af4d-8ff7a9b97ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:54.516592 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e39ee8-d82a-48a1-af4d-8ff7a9b97ec3 admin admin] PATH_INFO: `/v3/users/253bb08ca26f44b78f7e71d84820bd49` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:54.519473 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e39ee8-d82a-48a1-af4d-8ff7a9b97ec3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=253bb08ca26f44b78f7e71d84820bd49)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:54.523230 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e39ee8-d82a-48a1-af4d-8ff7a9b97ec3 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:54.660915 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-58e39ee8-d82a-48a1-af4d-8ff7a9b97ec3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '253bb08ca26f44b78f7e71d84820bd49'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:54.662327 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-58e39ee8-d82a-48a1-af4d-8ff7a9b97ec3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '253bb08ca26f44b78f7e71d84820bd49'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:54.663955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:54.690749 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-58e39ee8-d82a-48a1-af4d-8ff7a9b97ec3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '253bb08ca26f44b78f7e71d84820bd49'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:54.693374 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-58e39ee8-d82a-48a1-af4d-8ff7a9b97ec3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '253bb08ca26f44b78f7e71d84820bd49'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:54.729590 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-58e39ee8-d82a-48a1-af4d-8ff7a9b97ec3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '253bb08ca26f44b78f7e71d84820bd49'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:54.768478 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 936/1886] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:01:54 2022] DELETE /identity/v3/users/253bb08ca26f44b78f7e71d84820bd49 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:55.899149 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68880e68-8386-4a39-bcca-acc3caac52d8 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:55.900626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-68880e68-8386-4a39-bcca-acc3caac52d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:55.928635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68880e68-8386-4a39-bcca-acc3caac52d8 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:56.127433 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b93f0cf-ae21-4357-a309-2bf354a0d884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:56.128349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93f0cf-ae21-4357-a309-2bf354a0d884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:56.128640 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93f0cf-ae21-4357-a309-2bf354a0d884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:56.128978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b93f0cf-ae21-4357-a309-2bf354a0d884 admin admin] PATH_INFO: `/v3/projects/4c6af46c467b49d883798bdbd4ac01c0` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:56.130378 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93f0cf-ae21-4357-a309-2bf354a0d884 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c6af46c467b49d883798bdbd4ac01c0)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:56.132352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b93f0cf-ae21-4357-a309-2bf354a0d884 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:56.266132 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3b93f0cf-ae21-4357-a309-2bf354a0d884 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c6af46c467b49d883798bdbd4ac01c0', 'request_id': 'req-3b93f0cf-ae21-4357-a309-2bf354a0d884'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:56.281084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3b93f0cf-ae21-4357-a309-2bf354a0d884 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c6af46c467b49d883798bdbd4ac01c0', 'request_id': 'req-3b93f0cf-ae21-4357-a309-2bf354a0d884'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:56.291397 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3b93f0cf-ae21-4357-a309-2bf354a0d884 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c6af46c467b49d883798bdbd4ac01c0', 'request_id': 'req-3b93f0cf-ae21-4357-a309-2bf354a0d884'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:56.309291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3dbdd-1ce5-4189-b39c-944b26089b50 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:56.310376 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3dbdd-1ce5-4189-b39c-944b26089b50 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:56.310713 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b3dbdd-1ce5-4189-b39c-944b26089b50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:56.329333 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3b93f0cf-ae21-4357-a309-2bf354a0d884 admin admin] The token cache is being invalidate because project 4c6af46c467b49d883798bdbd4ac01c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:01:56.330936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3b93f0cf-ae21-4357-a309-2bf354a0d884 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:56.332677 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3b93f0cf-ae21-4357-a309-2bf354a0d884 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:56.334773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 951/1887] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:01:55 2022] DELETE /identity/v3/projects/4c6af46c467b49d883798bdbd4ac01c0 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:56.339895 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19850557-1a33-4002-8626-b72a4a3604fb None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:56.340429 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19850557-1a33-4002-8626-b72a4a3604fb None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:56.340945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19850557-1a33-4002-8626-b72a4a3604fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:56.361209 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-26b3dbdd-1ce5-4189-b39c-944b26089b50 None None] MFA Rules not processed for user `105edf55cdfc4085bde1ccfffddd325e`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:56.362539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-26b3dbdd-1ce5-4189-b39c-944b26089b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:56.374611 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-19850557-1a33-4002-8626-b72a4a3604fb None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:56.376427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-19850557-1a33-4002-8626-b72a4a3604fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:56.435076 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:56.488148 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 937/1888] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:56 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:01:56.496755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 952/1889] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:01:56.502867 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3dbdd-1ce5-4189-b39c-944b26089b50 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:56.504334 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-26b3dbdd-1ce5-4189-b39c-944b26089b50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:56.504764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990abd92-ff13-40f3-a68b-3dacd0ca5b03 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:56.505071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990abd92-ff13-40f3-a68b-3dacd0ca5b03 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:56.505363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990abd92-ff13-40f3-a68b-3dacd0ca5b03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:56.531820 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-990abd92-ff13-40f3-a68b-3dacd0ca5b03 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:56.533389 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-990abd92-ff13-40f3-a68b-3dacd0ca5b03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:56.545921 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 953/1890] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:01:56.551691 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990abd92-ff13-40f3-a68b-3dacd0ca5b03 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:56.563410 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990abd92-ff13-40f3-a68b-3dacd0ca5b03 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:56.573779 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af84bdc-8590-416c-9a1b-e1b327a03bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:56.575273 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af84bdc-8590-416c-9a1b-e1b327a03bc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:56.576359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af84bdc-8590-416c-9a1b-e1b327a03bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:56.576799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af84bdc-8590-416c-9a1b-e1b327a03bc5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:56.578099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af84bdc-8590-416c-9a1b-e1b327a03bc5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:01:56.578584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af84bdc-8590-416c-9a1b-e1b327a03bc5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:56.581190 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af84bdc-8590-416c-9a1b-e1b327a03bc5 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:56.599736 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b3dbdd-1ce5-4189-b39c-944b26089b50 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:56.600431 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 954/1891] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:01:56.601703 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af84bdc-8590-416c-9a1b-e1b327a03bc5 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:56.605659 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef70b1-eac4-41bc-ade5-5f72e3314661 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:56.606472 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef70b1-eac4-41bc-ade5-5f72e3314661 service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:56.606858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef70b1-eac4-41bc-ade5-5f72e3314661 service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:56.611073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef70b1-eac4-41bc-ade5-5f72e3314661 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:56.614756 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af84bdc-8590-416c-9a1b-e1b327a03bc5 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:56.615327 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-aeef70b1-eac4-41bc-ade5-5f72e3314661 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:56.621876 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30292b-ac80-475c-a619-db5ba1a4527d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:56.623600 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30292b-ac80-475c-a619-db5ba1a4527d admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:56.624132 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30292b-ac80-475c-a619-db5ba1a4527d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:56.624555 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30292b-ac80-475c-a619-db5ba1a4527d admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:56.625452 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30292b-ac80-475c-a619-db5ba1a4527d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:56.628313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30292b-ac80-475c-a619-db5ba1a4527d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:56.676718 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 955/1892] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:01:56 2022] POST /identity/v3/projects => generated 407 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:56.680871 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30292b-ac80-475c-a619-db5ba1a4527d admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:56.698212 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:56.717088 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef70b1-eac4-41bc-ade5-5f72e3314661 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:56.719792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef70b1-eac4-41bc-ade5-5f72e3314661 service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:56.743027 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 938/1893] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:56.789100 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30292b-ac80-475c-a619-db5ba1a4527d admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:56.796276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a983c-ce86-4d8f-a1ad-29de5dfd3128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:56.796947 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a983c-ce86-4d8f-a1ad-29de5dfd3128 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:56.797286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a983c-ce86-4d8f-a1ad-29de5dfd3128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:56.798201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a983c-ce86-4d8f-a1ad-29de5dfd3128 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:56.799028 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a983c-ce86-4d8f-a1ad-29de5dfd3128 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:56.802393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a983c-ce86-4d8f-a1ad-29de5dfd3128 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:56.836689 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 956/1894] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:01:56 2022] POST /identity/v3/users => generated 304 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:01:56.839269 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef70b1-eac4-41bc-ade5-5f72e3314661 service nova] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:56.853413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef70b1-eac4-41bc-ade5-5f72e3314661 service nova] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:56.864797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006c00b-a727-45ca-9880-b77fbf72d230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:56.866181 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006c00b-a727-45ca-9880-b77fbf72d230 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:56.866546 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006c00b-a727-45ca-9880-b77fbf72d230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:56.867451 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006c00b-a727-45ca-9880-b77fbf72d230 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:56.868714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006c00b-a727-45ca-9880-b77fbf72d230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:56.870936 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9006c00b-a727-45ca-9880-b77fbf72d230 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:56.902211 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a983c-ce86-4d8f-a1ad-29de5dfd3128 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:56.903040 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 939/1895] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:56.915289 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a983c-ce86-4d8f-a1ad-29de5dfd3128 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:56.920330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cad4c1-915e-4a0b-824c-649f86d9d67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:56.921281 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cad4c1-915e-4a0b-824c-649f86d9d67e admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:56.921578 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cad4c1-915e-4a0b-824c-649f86d9d67e admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:56.921954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cad4c1-915e-4a0b-824c-649f86d9d67e admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:56.922805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cad4c1-915e-4a0b-824c-649f86d9d67e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:56.931703 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cad4c1-915e-4a0b-824c-649f86d9d67e admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:56.953965 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 957/1896] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:01:56.955708 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006c00b-a727-45ca-9880-b77fbf72d230 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:56.969914 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006c00b-a727-45ca-9880-b77fbf72d230 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:56.974983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c13769-4907-42c5-8547-3ef09a330939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:56.975908 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c13769-4907-42c5-8547-3ef09a330939 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:56.976191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c13769-4907-42c5-8547-3ef09a330939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:56.976487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c13769-4907-42c5-8547-3ef09a330939 admin admin] PATH_INFO: `/v3/projects/dd9386437d4e4c2fa3ef654c7a9b6c66/users/c1855fdefd3d48ae9f5e16c3a2f08f07/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:56.997682 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c13769-4907-42c5-8547-3ef09a330939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd9386437d4e4c2fa3ef654c7a9b6c66, user_id=c1855fdefd3d48ae9f5e16c3a2f08f07, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:57.000795 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c13769-4907-42c5-8547-3ef09a330939 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:57.022875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 940/1897] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:56 2022] PUT /identity/v3/projects/dd9386437d4e4c2fa3ef654c7a9b6c66/users/c1855fdefd3d48ae9f5e16c3a2f08f07/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:57.031857 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8067c-cbc9-4083-8599-e81b051c8fcf None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:57.032255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8067c-cbc9-4083-8599-e81b051c8fcf None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:57.032591 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd8067c-cbc9-4083-8599-e81b051c8fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:57.080750 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-cbd8067c-cbc9-4083-8599-e81b051c8fcf None None] MFA Rules not processed for user `c1855fdefd3d48ae9f5e16c3a2f08f07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:57.082159 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-cbd8067c-cbc9-4083-8599-e81b051c8fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:57.208398 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 958/1898] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:57 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:57.212130 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c13769-4907-42c5-8547-3ef09a330939 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:57.352554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c13769-4907-42c5-8547-3ef09a330939 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:57.360131 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f885dcb-c86f-4b43-814d-254d36439bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:57.361036 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f885dcb-c86f-4b43-814d-254d36439bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:57.361429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f885dcb-c86f-4b43-814d-254d36439bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:57.361938 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f885dcb-c86f-4b43-814d-254d36439bf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:57.362968 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f885dcb-c86f-4b43-814d-254d36439bf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:57.367066 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f885dcb-c86f-4b43-814d-254d36439bf2 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:57.420882 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 941/1899] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:01:57 2022] POST /identity/v3/projects => generated 409 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:01:57.424924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8067c-cbc9-4083-8599-e81b051c8fcf None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:57.435409 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:57.440990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd8067c-cbc9-4083-8599-e81b051c8fcf None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:57.535552 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae98df8-0e2c-4d59-a730-6cb90417b248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:57.537762 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae98df8-0e2c-4d59-a730-6cb90417b248 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:57.538071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae98df8-0e2c-4d59-a730-6cb90417b248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:57.539991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae98df8-0e2c-4d59-a730-6cb90417b248 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:57.550846 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae98df8-0e2c-4d59-a730-6cb90417b248 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:57.550846 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae98df8-0e2c-4d59-a730-6cb90417b248 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:57.584171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 959/1900] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:01:57 2022] POST /identity/v3/users => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:01:57.585147 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f885dcb-c86f-4b43-814d-254d36439bf2 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:57.600656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f885dcb-c86f-4b43-814d-254d36439bf2 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:57.607858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccd30d-b6d4-4c7f-89b4-108e0c652637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:57.610177 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccd30d-b6d4-4c7f-89b4-108e0c652637 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:57.611044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccd30d-b6d4-4c7f-89b4-108e0c652637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:57.611601 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ccd30d-b6d4-4c7f-89b4-108e0c652637 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:57.612995 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccd30d-b6d4-4c7f-89b4-108e0c652637 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:57.616003 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ccd30d-b6d4-4c7f-89b4-108e0c652637 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:57.637961 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 942/1901] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:01:57.639959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae98df8-0e2c-4d59-a730-6cb90417b248 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:57.655640 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae98df8-0e2c-4d59-a730-6cb90417b248 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:57.661460 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839dbe1-9154-4f3b-bfb5-845c55f195db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:57.662845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839dbe1-9154-4f3b-bfb5-845c55f195db admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:57.663141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839dbe1-9154-4f3b-bfb5-845c55f195db admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:57.663464 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e839dbe1-9154-4f3b-bfb5-845c55f195db admin admin] PATH_INFO: `/v3/projects/4570a205938b40029454385e7940d5e8/users/a6633781fb52435f9ce99ad7912617dd/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:57.684859 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e839dbe1-9154-4f3b-bfb5-845c55f195db admin admin] RBAC: Authorizing `identity:create_grant(project_id=4570a205938b40029454385e7940d5e8, user_id=a6633781fb52435f9ce99ad7912617dd, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:57.688620 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e839dbe1-9154-4f3b-bfb5-845c55f195db admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:57.719036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 960/1902] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:57 2022] PUT /identity/v3/projects/4570a205938b40029454385e7940d5e8/users/a6633781fb52435f9ce99ad7912617dd/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:01:57.719930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccd30d-b6d4-4c7f-89b4-108e0c652637 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:57.740081 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ccd30d-b6d4-4c7f-89b4-108e0c652637 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:57.832006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9441dfd4-819e-4793-989d-37cd84c51703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:57.832843 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441dfd4-819e-4793-989d-37cd84c51703 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:57.833136 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441dfd4-819e-4793-989d-37cd84c51703 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:57.833419 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9441dfd4-819e-4793-989d-37cd84c51703 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:57.834215 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441dfd4-819e-4793-989d-37cd84c51703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:57.835893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9441dfd4-819e-4793-989d-37cd84c51703 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:57.851140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 943/1903] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:01:57.855330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839dbe1-9154-4f3b-bfb5-845c55f195db admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:57.868947 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e839dbe1-9154-4f3b-bfb5-845c55f195db admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:57.881822 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d4622-09ef-4db9-a331-ec99ac1f6ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:57.883030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d4622-09ef-4db9-a331-ec99ac1f6ed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:57.883477 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d4622-09ef-4db9-a331-ec99ac1f6ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:57.883761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0d4622-09ef-4db9-a331-ec99ac1f6ed1 admin admin] PATH_INFO: `/v3/projects/4570a205938b40029454385e7940d5e8/users/a6633781fb52435f9ce99ad7912617dd/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:57.888796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d4622-09ef-4db9-a331-ec99ac1f6ed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4570a205938b40029454385e7940d5e8, user_id=a6633781fb52435f9ce99ad7912617dd, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:57.891634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0d4622-09ef-4db9-a331-ec99ac1f6ed1 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:57.906395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 961/1904] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:01:57 2022] PUT /identity/v3/projects/4570a205938b40029454385e7940d5e8/users/a6633781fb52435f9ce99ad7912617dd/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:01:57.917573 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90783068-b370-4775-9f47-9038ee696bd4 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:57.917894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90783068-b370-4775-9f47-9038ee696bd4 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:57.918182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90783068-b370-4775-9f47-9038ee696bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:57.957080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-90783068-b370-4775-9f47-9038ee696bd4 None None] MFA Rules not processed for user `a6633781fb52435f9ce99ad7912617dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:57.958409 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-90783068-b370-4775-9f47-9038ee696bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:57.976913 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:58.058451 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 944/1905] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:57 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:58.069954 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d4622-09ef-4db9-a331-ec99ac1f6ed1 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:58.072636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-bb0d4622-09ef-4db9-a331-ec99ac1f6ed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:58.151979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0d4622-09ef-4db9-a331-ec99ac1f6ed1 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:58.156901 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5326013-16f6-4d56-9a45-f20dad66395d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:58.157835 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5326013-16f6-4d56-9a45-f20dad66395d service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:58.158296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5326013-16f6-4d56-9a45-f20dad66395d service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:58.158784 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5326013-16f6-4d56-9a45-f20dad66395d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:58.172278 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5326013-16f6-4d56-9a45-f20dad66395d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:58.174456 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5326013-16f6-4d56-9a45-f20dad66395d service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:58.191616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 962/1906] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:58 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:01:58.392635 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90783068-b370-4775-9f47-9038ee696bd4 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:58.408410 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90783068-b370-4775-9f47-9038ee696bd4 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:58.421079 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99532ca8-a405-4c0a-88b7-7f9c1296d5b2 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:58.421384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99532ca8-a405-4c0a-88b7-7f9c1296d5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:58.421668 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99532ca8-a405-4c0a-88b7-7f9c1296d5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:58.421974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf150e3-585a-498c-9319-fdb80238f7d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:58.422746 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf150e3-585a-498c-9319-fdb80238f7d3 service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:58.423040 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf150e3-585a-498c-9319-fdb80238f7d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:58.423325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf150e3-585a-498c-9319-fdb80238f7d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:58.442053 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-99532ca8-a405-4c0a-88b7-7f9c1296d5b2 None None] MFA Rules not processed for user `01dd0e5661f44d05a01d7a5e82dbadf3`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:58.443382 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-99532ca8-a405-4c0a-88b7-7f9c1296d5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:58.490291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf150e3-585a-498c-9319-fdb80238f7d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:58.491782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf150e3-585a-498c-9319-fdb80238f7d3 service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:58.501322 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 945/1907] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:58 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:01:58.521074 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 963/1908] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:01:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:59.576538 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6db06c-e0d1-4ace-b99f-cac133477305 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:59.577572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6db06c-e0d1-4ace-b99f-cac133477305 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:59.578227 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6db06c-e0d1-4ace-b99f-cac133477305 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:59.580422 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 946/1909] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 23:01:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 23:01:59.625422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fae721-1173-48c7-9c71-a0e5ea9dd785 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:59.626324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fae721-1173-48c7-9c71-a0e5ea9dd785 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:59.626916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fae721-1173-48c7-9c71-a0e5ea9dd785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:59.636933 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:59.652420 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-c3fae721-1173-48c7-9c71-a0e5ea9dd785 None None] MFA Rules not processed for user `f9593939641c4c7797c1981206b40e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:01:59.654573 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c3fae721-1173-48c7-9c71-a0e5ea9dd785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:59.681878 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6db06c-e0d1-4ace-b99f-cac133477305 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:59.683786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6d6db06c-e0d1-4ace-b99f-cac133477305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:59.773430 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 964/1910] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 23:01:59 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:01:59.788273 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6db06c-e0d1-4ace-b99f-cac133477305 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:59.788479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fae721-1173-48c7-9c71-a0e5ea9dd785 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:01:59.789701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c3fae721-1173-48c7-9c71-a0e5ea9dd785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:01:59.795294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea93d27-9143-497c-9f94-0fedc77c4bc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:59.796431 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea93d27-9143-497c-9f94-0fedc77c4bc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:59.796859 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea93d27-9143-497c-9f94-0fedc77c4bc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:59.797214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea93d27-9143-497c-9f94-0fedc77c4bc7 admin admin] PATH_INFO: `/v3/users/a4a6c4623c5f471b8241257180b64d37` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:59.798541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea93d27-9143-497c-9f94-0fedc77c4bc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a4a6c4623c5f471b8241257180b64d37)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:59.800535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea93d27-9143-497c-9f94-0fedc77c4bc7 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:59.806083 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fae721-1173-48c7-9c71-a0e5ea9dd785 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:01:59.812016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9fea53-c427-4875-b405-506f59b2a481 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9593939641c4c7797c1981206b40e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:01:59.812678 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fea53-c427-4875-b405-506f59b2a481 service placement] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:01:59.812973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fea53-c427-4875-b405-506f59b2a481 service placement] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:01:59.813257 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9fea53-c427-4875-b405-506f59b2a481 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:01:59.822013 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9fea53-c427-4875-b405-506f59b2a481 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:01:59.824444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9fea53-c427-4875-b405-506f59b2a481 service placement] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:01:59.838824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 965/1911] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:01:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:01:59.866189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-dea93d27-9143-497c-9f94-0fedc77c4bc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a4a6c4623c5f471b8241257180b64d37'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:59.867023 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-dea93d27-9143-497c-9f94-0fedc77c4bc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a4a6c4623c5f471b8241257180b64d37'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:59.868551 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:01:59.887065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-dea93d27-9143-497c-9f94-0fedc77c4bc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a4a6c4623c5f471b8241257180b64d37'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:59.887841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-dea93d27-9143-497c-9f94-0fedc77c4bc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a4a6c4623c5f471b8241257180b64d37'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:59.912144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-dea93d27-9143-497c-9f94-0fedc77c4bc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a4a6c4623c5f471b8241257180b64d37'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:01:59.933086 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 947/1912] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:01:59 2022] DELETE /identity/v3/users/a4a6c4623c5f471b8241257180b64d37 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:00.139106 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9fea53-c427-4875-b405-506f59b2a481 service placement] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:00.153668 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9fea53-c427-4875-b405-506f59b2a481 service placement] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:00.225428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a82bb5-384b-4d41-a94f-decdeaababe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:00.226378 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a82bb5-384b-4d41-a94f-decdeaababe3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:00.226736 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a82bb5-384b-4d41-a94f-decdeaababe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:00.227141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a82bb5-384b-4d41-a94f-decdeaababe3 admin admin] PATH_INFO: `/v3/projects/c2300d78ad8244e3be17c1f388603de7` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:00.230642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a82bb5-384b-4d41-a94f-decdeaababe3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2300d78ad8244e3be17c1f388603de7)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:00.230779 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a82bb5-384b-4d41-a94f-decdeaababe3 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:00.323601 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-00a82bb5-384b-4d41-a94f-decdeaababe3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2300d78ad8244e3be17c1f388603de7', 'request_id': 'req-00a82bb5-384b-4d41-a94f-decdeaababe3'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:00.330438 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-00a82bb5-384b-4d41-a94f-decdeaababe3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2300d78ad8244e3be17c1f388603de7', 'request_id': 'req-00a82bb5-384b-4d41-a94f-decdeaababe3'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:00.336532 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-00a82bb5-384b-4d41-a94f-decdeaababe3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2300d78ad8244e3be17c1f388603de7', 'request_id': 'req-00a82bb5-384b-4d41-a94f-decdeaababe3'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:00.350993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-00a82bb5-384b-4d41-a94f-decdeaababe3 admin admin] The token cache is being invalidate because project c2300d78ad8244e3be17c1f388603de7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:00.351284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-00a82bb5-384b-4d41-a94f-decdeaababe3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:00.351926 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-00a82bb5-384b-4d41-a94f-decdeaababe3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:00.360382 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 966/1913] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:00 2022] DELETE /identity/v3/projects/c2300d78ad8244e3be17c1f388603de7 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:00.363871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9f73e-0397-4697-aec8-136caa684fcf None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:00.364250 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9f73e-0397-4697-aec8-136caa684fcf None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:00.364802 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f9f73e-0397-4697-aec8-136caa684fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:00.406866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-76f9f73e-0397-4697-aec8-136caa684fcf None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:00.408114 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-76f9f73e-0397-4697-aec8-136caa684fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:00.495747 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 948/1914] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:00.503328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d52074-b34c-4621-8584-1aaabea43f74 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:00.503655 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d52074-b34c-4621-8584-1aaabea43f74 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:00.503955 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d52074-b34c-4621-8584-1aaabea43f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:00.527697 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-64d52074-b34c-4621-8584-1aaabea43f74 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:00.529720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-64d52074-b34c-4621-8584-1aaabea43f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:00.539600 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 967/1915] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:00.545176 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9f73e-0397-4697-aec8-136caa684fcf None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:00.559762 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f9f73e-0397-4697-aec8-136caa684fcf None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:00.561216 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f4a22-03b5-491f-b52e-d3e0219297fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:00.561908 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f4a22-03b5-491f-b52e-d3e0219297fe admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:00.562233 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f4a22-03b5-491f-b52e-d3e0219297fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:00.562518 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f4a22-03b5-491f-b52e-d3e0219297fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:00.563308 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f4a22-03b5-491f-b52e-d3e0219297fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:02:00.563626 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f4a22-03b5-491f-b52e-d3e0219297fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:00.565958 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3f4a22-03b5-491f-b52e-d3e0219297fe admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:00.588303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 949/1916] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:02:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:02:00.595313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d52074-b34c-4621-8584-1aaabea43f74 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:00.608762 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d52074-b34c-4621-8584-1aaabea43f74 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:00.619739 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641373f-9ffc-42f7-874d-201b50f2db74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:00.620528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641373f-9ffc-42f7-874d-201b50f2db74 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:00.621417 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641373f-9ffc-42f7-874d-201b50f2db74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:00.621776 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641373f-9ffc-42f7-874d-201b50f2db74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:00.622659 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641373f-9ffc-42f7-874d-201b50f2db74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:00.631206 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641373f-9ffc-42f7-874d-201b50f2db74 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:00.649263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:00.675001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 968/1917] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:00 2022] POST /identity/v3/projects => generated 365 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:00.677113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f4a22-03b5-491f-b52e-d3e0219297fe admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:00.690066 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f4a22-03b5-491f-b52e-d3e0219297fe admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:00.774947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b149d-b708-433f-a79b-a5ddee130fef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:00.776029 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b149d-b708-433f-a79b-a5ddee130fef admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:00.776415 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b149d-b708-433f-a79b-a5ddee130fef admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:00.776944 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4b149d-b708-433f-a79b-a5ddee130fef admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:00.780116 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b149d-b708-433f-a79b-a5ddee130fef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:00.782514 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4b149d-b708-433f-a79b-a5ddee130fef admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:00.809249 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 950/1918] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:00 2022] POST /identity/v3/users => generated 283 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:00.814948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641373f-9ffc-42f7-874d-201b50f2db74 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:00.835425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641373f-9ffc-42f7-874d-201b50f2db74 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:00.840557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb91727-139e-4520-a7b0-456befc09198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:00.842037 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb91727-139e-4520-a7b0-456befc09198 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:00.842484 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb91727-139e-4520-a7b0-456befc09198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:00.842952 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb91727-139e-4520-a7b0-456befc09198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:00.843845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb91727-139e-4520-a7b0-456befc09198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:00.845600 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb91727-139e-4520-a7b0-456befc09198 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:00.874179 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 969/1919] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:00.875136 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b149d-b708-433f-a79b-a5ddee130fef admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:00.893937 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4b149d-b708-433f-a79b-a5ddee130fef admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:00.899248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ac76ec-5d1c-4652-8b14-53a19d32b064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:00.899965 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac76ec-5d1c-4652-8b14-53a19d32b064 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:00.900257 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac76ec-5d1c-4652-8b14-53a19d32b064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:00.900540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ac76ec-5d1c-4652-8b14-53a19d32b064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:00.901275 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac76ec-5d1c-4652-8b14-53a19d32b064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:00.903004 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ac76ec-5d1c-4652-8b14-53a19d32b064 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:00.929532 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 951/1920] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:00 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:00.931109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb91727-139e-4520-a7b0-456befc09198 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:00.951734 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb91727-139e-4520-a7b0-456befc09198 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:00.960692 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb4f4c-9803-4c9a-884c-431e2bdc154f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:00.962140 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb4f4c-9803-4c9a-884c-431e2bdc154f admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:00.962628 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb4f4c-9803-4c9a-884c-431e2bdc154f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:00.963077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb4f4c-9803-4c9a-884c-431e2bdc154f admin admin] PATH_INFO: `/v3/projects/390df83726124500bde5f207ee4b6629/users/e5770101917a49269e6850a2003071c5/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:00.986757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb4f4c-9803-4c9a-884c-431e2bdc154f admin admin] RBAC: Authorizing `identity:create_grant(project_id=390df83726124500bde5f207ee4b6629, user_id=e5770101917a49269e6850a2003071c5, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:00.989418 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bb4f4c-9803-4c9a-884c-431e2bdc154f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:01.013919 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 970/1921] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:00 2022] PUT /identity/v3/projects/390df83726124500bde5f207ee4b6629/users/e5770101917a49269e6850a2003071c5/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:01.024459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3b9f8-8168-4c84-8ff5-df08e96405c2 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:01.024856 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3b9f8-8168-4c84-8ff5-df08e96405c2 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:01.025210 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d3b9f8-8168-4c84-8ff5-df08e96405c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:01.065438 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-79d3b9f8-8168-4c84-8ff5-df08e96405c2 None None] MFA Rules not processed for user `e5770101917a49269e6850a2003071c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:01.067817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-79d3b9f8-8168-4c84-8ff5-df08e96405c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:01.087355 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:01.151486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 952/1922] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:01.164133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb4f4c-9803-4c9a-884c-431e2bdc154f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:01.166106 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-10bb4f4c-9803-4c9a-884c-431e2bdc154f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:01.245878 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb4f4c-9803-4c9a-884c-431e2bdc154f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:01.250185 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff090a2-49b2-435e-b3aa-ec8d0d062071 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:01.250813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff090a2-49b2-435e-b3aa-ec8d0d062071 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:01.251314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff090a2-49b2-435e-b3aa-ec8d0d062071 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:01.251756 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff090a2-49b2-435e-b3aa-ec8d0d062071 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:01.261472 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff090a2-49b2-435e-b3aa-ec8d0d062071 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:01.263178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff090a2-49b2-435e-b3aa-ec8d0d062071 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:01.275446 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 971/1923] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:01 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:05.552132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925827b-320f-4690-8fdf-13ad66a61bfe None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:05.552641 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925827b-320f-4690-8fdf-13ad66a61bfe None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:05.552744 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8925827b-320f-4690-8fdf-13ad66a61bfe None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:05.554366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 953/1924] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 23:02:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 23:02:05.565717 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bed4ae-d6ac-4141-bf38-6a882bba13e3 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:05.566661 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bed4ae-d6ac-4141-bf38-6a882bba13e3 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:05.567359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bed4ae-d6ac-4141-bf38-6a882bba13e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:05.593090 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-b0bed4ae-d6ac-4141-bf38-6a882bba13e3 None None] MFA Rules not processed for user `f9593939641c4c7797c1981206b40e7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:05.594363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b0bed4ae-d6ac-4141-bf38-6a882bba13e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:05.667368 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 972/1925] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 23:02:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:05.678955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925827b-320f-4690-8fdf-13ad66a61bfe None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:05.680983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-8925827b-320f-4690-8fdf-13ad66a61bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:05.697974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8925827b-320f-4690-8fdf-13ad66a61bfe None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:05.703459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd6683-3ef0-4ddd-bc46-747d930dcc3b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9593939641c4c7797c1981206b40e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:05.704117 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd6683-3ef0-4ddd-bc46-747d930dcc3b service placement] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:05.704405 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd6683-3ef0-4ddd-bc46-747d930dcc3b service placement] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:05.704687 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dd6683-3ef0-4ddd-bc46-747d930dcc3b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:05.713366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd6683-3ef0-4ddd-bc46-747d930dcc3b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:05.714987 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dd6683-3ef0-4ddd-bc46-747d930dcc3b service placement] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:05.725452 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 954/1926] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:06.171864 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bed4ae-d6ac-4141-bf38-6a882bba13e3 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:06.173444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b0bed4ae-d6ac-4141-bf38-6a882bba13e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:06.236068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:06.251490 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bed4ae-d6ac-4141-bf38-6a882bba13e3 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:06.256243 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec2028-f485-434c-a497-812e6bc8396f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:06.257161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec2028-f485-434c-a497-812e6bc8396f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:06.257449 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec2028-f485-434c-a497-812e6bc8396f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:06.257732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec2028-f485-434c-a497-812e6bc8396f admin admin] PATH_INFO: `/v3/users/ddc4a97488674b52a9b68a408ab2a5a9` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:06.258984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec2028-f485-434c-a497-812e6bc8396f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddc4a97488674b52a9b68a408ab2a5a9)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:06.260907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec2028-f485-434c-a497-812e6bc8396f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:06.319726 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bcec2028-f485-434c-a497-812e6bc8396f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddc4a97488674b52a9b68a408ab2a5a9'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:06.327442 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bcec2028-f485-434c-a497-812e6bc8396f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddc4a97488674b52a9b68a408ab2a5a9'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:06.328157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bcec2028-f485-434c-a497-812e6bc8396f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddc4a97488674b52a9b68a408ab2a5a9'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:06.342413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bcec2028-f485-434c-a497-812e6bc8396f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddc4a97488674b52a9b68a408ab2a5a9'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:06.343233 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bcec2028-f485-434c-a497-812e6bc8396f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddc4a97488674b52a9b68a408ab2a5a9'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:06.358498 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 973/1927] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:06 2022] DELETE /identity/v3/users/ddc4a97488674b52a9b68a408ab2a5a9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:06.551971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd6683-3ef0-4ddd-bc46-747d930dcc3b service placement] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:06.563719 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dd6683-3ef0-4ddd-bc46-747d930dcc3b service placement] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:06.663936 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade13ac-2283-457c-9348-76d2c6224231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:06.665023 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade13ac-2283-457c-9348-76d2c6224231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:06.665387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade13ac-2283-457c-9348-76d2c6224231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:06.665820 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cade13ac-2283-457c-9348-76d2c6224231 admin admin] PATH_INFO: `/v3/projects/5a1cfcacb5ed448cb73ad5054ed0964b` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:06.667409 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade13ac-2283-457c-9348-76d2c6224231 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a1cfcacb5ed448cb73ad5054ed0964b)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:06.670819 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cade13ac-2283-457c-9348-76d2c6224231 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:06.753180 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cade13ac-2283-457c-9348-76d2c6224231 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a1cfcacb5ed448cb73ad5054ed0964b', 'request_id': 'req-cade13ac-2283-457c-9348-76d2c6224231'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:06.764830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cade13ac-2283-457c-9348-76d2c6224231 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a1cfcacb5ed448cb73ad5054ed0964b', 'request_id': 'req-cade13ac-2283-457c-9348-76d2c6224231'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:06.770481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cade13ac-2283-457c-9348-76d2c6224231 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a1cfcacb5ed448cb73ad5054ed0964b', 'request_id': 'req-cade13ac-2283-457c-9348-76d2c6224231'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:06.787672 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cade13ac-2283-457c-9348-76d2c6224231 admin admin] The token cache is being invalidate because project 5a1cfcacb5ed448cb73ad5054ed0964b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:06.787962 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cade13ac-2283-457c-9348-76d2c6224231 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:06.788612 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cade13ac-2283-457c-9348-76d2c6224231 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:06.790312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 955/1928] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:06 2022] DELETE /identity/v3/projects/5a1cfcacb5ed448cb73ad5054ed0964b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:06.794376 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec2028-f485-434c-a497-812e6bc8396f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:06.795845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-bcec2028-f485-434c-a497-812e6bc8396f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:06.810207 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec2028-f485-434c-a497-812e6bc8396f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:06.883834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8db9b8-1929-4736-a868-cd3c43574a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:06.884994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8db9b8-1929-4736-a868-cd3c43574a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:06.885284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8db9b8-1929-4736-a868-cd3c43574a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:06.885563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8db9b8-1929-4736-a868-cd3c43574a74 admin admin] PATH_INFO: `/v3/users/544705b920b141fcacfe9fa0337fbae4` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:06.886828 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8db9b8-1929-4736-a868-cd3c43574a74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=544705b920b141fcacfe9fa0337fbae4)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:06.889417 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8db9b8-1929-4736-a868-cd3c43574a74 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:06.942920 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-de8db9b8-1929-4736-a868-cd3c43574a74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '544705b920b141fcacfe9fa0337fbae4'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:06.950808 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-de8db9b8-1929-4736-a868-cd3c43574a74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '544705b920b141fcacfe9fa0337fbae4'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:06.951719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-de8db9b8-1929-4736-a868-cd3c43574a74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '544705b920b141fcacfe9fa0337fbae4'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:06.964374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-de8db9b8-1929-4736-a868-cd3c43574a74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '544705b920b141fcacfe9fa0337fbae4'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:06.965196 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-de8db9b8-1929-4736-a868-cd3c43574a74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '544705b920b141fcacfe9fa0337fbae4'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:06.967511 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:06.980515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 974/1929] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:06 2022] DELETE /identity/v3/users/544705b920b141fcacfe9fa0337fbae4 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:07.168758 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade13ac-2283-457c-9348-76d2c6224231 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:07.170659 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-cade13ac-2283-457c-9348-76d2c6224231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:07.190049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cade13ac-2283-457c-9348-76d2c6224231 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:07.269843 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:07.303083 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdfc49a-4ea6-48b7-9177-fe2e5fc28765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:07.304170 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdfc49a-4ea6-48b7-9177-fe2e5fc28765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:07.304469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdfc49a-4ea6-48b7-9177-fe2e5fc28765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:07.306235 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bdfc49a-4ea6-48b7-9177-fe2e5fc28765 admin admin] PATH_INFO: `/v3/projects/01a566c0abd24253857e5926c1f653b5` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:07.306235 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdfc49a-4ea6-48b7-9177-fe2e5fc28765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01a566c0abd24253857e5926c1f653b5)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:07.308017 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bdfc49a-4ea6-48b7-9177-fe2e5fc28765 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:07.377057 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-8bdfc49a-4ea6-48b7-9177-fe2e5fc28765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01a566c0abd24253857e5926c1f653b5', 'request_id': 'req-8bdfc49a-4ea6-48b7-9177-fe2e5fc28765'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:07.385079 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-8bdfc49a-4ea6-48b7-9177-fe2e5fc28765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01a566c0abd24253857e5926c1f653b5', 'request_id': 'req-8bdfc49a-4ea6-48b7-9177-fe2e5fc28765'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:07.391792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-8bdfc49a-4ea6-48b7-9177-fe2e5fc28765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01a566c0abd24253857e5926c1f653b5', 'request_id': 'req-8bdfc49a-4ea6-48b7-9177-fe2e5fc28765'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:07.405556 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-8bdfc49a-4ea6-48b7-9177-fe2e5fc28765 admin admin] The token cache is being invalidate because project 01a566c0abd24253857e5926c1f653b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:07.405839 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-8bdfc49a-4ea6-48b7-9177-fe2e5fc28765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:07.406508 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-8bdfc49a-4ea6-48b7-9177-fe2e5fc28765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:07.408212 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 956/1930] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:07 2022] DELETE /identity/v3/projects/01a566c0abd24253857e5926c1f653b5 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:07.414034 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b88b6-1173-4f1d-9ccc-cbd7a68d8a17 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:07.414334 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b88b6-1173-4f1d-9ccc-cbd7a68d8a17 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:07.414637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009b88b6-1173-4f1d-9ccc-cbd7a68d8a17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:07.434703 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-009b88b6-1173-4f1d-9ccc-cbd7a68d8a17 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:07.435919 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-009b88b6-1173-4f1d-9ccc-cbd7a68d8a17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:07.443074 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdfc49a-4ea6-48b7-9177-fe2e5fc28765 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:07.444535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-8bdfc49a-4ea6-48b7-9177-fe2e5fc28765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:07.500039 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 975/1931] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:07.508756 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463151d-50bb-4744-a249-4c9352579ec2 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:07.509056 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463151d-50bb-4744-a249-4c9352579ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:07.509343 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a463151d-50bb-4744-a249-4c9352579ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:07.511545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bdfc49a-4ea6-48b7-9177-fe2e5fc28765 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:07.517132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bae72d-772c-4e61-8b7e-69e7e4f58662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:07.518304 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bae72d-772c-4e61-8b7e-69e7e4f58662 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:07.518755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bae72d-772c-4e61-8b7e-69e7e4f58662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:07.519087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bae72d-772c-4e61-8b7e-69e7e4f58662 admin admin] PATH_INFO: `/v3/users/c1855fdefd3d48ae9f5e16c3a2f08f07` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:07.520733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bae72d-772c-4e61-8b7e-69e7e4f58662 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1855fdefd3d48ae9f5e16c3a2f08f07)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:07.523179 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bae72d-772c-4e61-8b7e-69e7e4f58662 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:07.533546 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a463151d-50bb-4744-a249-4c9352579ec2 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:07.535016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a463151d-50bb-4744-a249-4c9352579ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:07.544437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 976/1932] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:07.548810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463151d-50bb-4744-a249-4c9352579ec2 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:07.558111 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a463151d-50bb-4744-a249-4c9352579ec2 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:07.564063 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a019cc-1b6d-49fa-b21d-23853e7a0daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:07.564741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a019cc-1b6d-49fa-b21d-23853e7a0daa admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:07.565034 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a019cc-1b6d-49fa-b21d-23853e7a0daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:07.565325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a019cc-1b6d-49fa-b21d-23853e7a0daa admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:07.566092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a019cc-1b6d-49fa-b21d-23853e7a0daa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:02:07.566408 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a019cc-1b6d-49fa-b21d-23853e7a0daa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:07.568056 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a019cc-1b6d-49fa-b21d-23853e7a0daa admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:07.585868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 977/1933] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:02:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 23:02:07.586040 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-34bae72d-772c-4e61-8b7e-69e7e4f58662 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1855fdefd3d48ae9f5e16c3a2f08f07'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:07.586040 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-34bae72d-772c-4e61-8b7e-69e7e4f58662 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1855fdefd3d48ae9f5e16c3a2f08f07'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:07.587199 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a019cc-1b6d-49fa-b21d-23853e7a0daa admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:07.589001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-35a019cc-1b6d-49fa-b21d-23853e7a0daa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:07.592481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-34bae72d-772c-4e61-8b7e-69e7e4f58662 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1855fdefd3d48ae9f5e16c3a2f08f07'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:07.593215 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-34bae72d-772c-4e61-8b7e-69e7e4f58662 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1855fdefd3d48ae9f5e16c3a2f08f07'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:07.601780 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a019cc-1b6d-49fa-b21d-23853e7a0daa admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:07.604614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:07.607707 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5e32f-8b04-4fdc-bd77-52e59ec56511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:07.607870 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-34bae72d-772c-4e61-8b7e-69e7e4f58662 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1855fdefd3d48ae9f5e16c3a2f08f07'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:07.608372 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5e32f-8b04-4fdc-bd77-52e59ec56511 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:07.608665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5e32f-8b04-4fdc-bd77-52e59ec56511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:07.608951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b5e32f-8b04-4fdc-bd77-52e59ec56511 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:07.609642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5e32f-8b04-4fdc-bd77-52e59ec56511 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:07.612135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b5e32f-8b04-4fdc-bd77-52e59ec56511 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:07.627523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 957/1934] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:07 2022] DELETE /identity/v3/users/c1855fdefd3d48ae9f5e16c3a2f08f07 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:02:07.631853 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680de95-0a4d-43b3-ae9c-a2bb4fdd669a None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:07.632216 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680de95-0a4d-43b3-ae9c-a2bb4fdd669a None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:07.632490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6680de95-0a4d-43b3-ae9c-a2bb4fdd669a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:07.641857 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 978/1935] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:07 2022] POST /identity/v3/projects => generated 381 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:07.645920 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5e32f-8b04-4fdc-bd77-52e59ec56511 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:07.656798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-6680de95-0a4d-43b3-ae9c-a2bb4fdd669a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:07.658271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6680de95-0a4d-43b3-ae9c-a2bb4fdd669a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:07.719508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b5e32f-8b04-4fdc-bd77-52e59ec56511 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:07.722046 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 958/1936] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:07.724609 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6edf3-4402-4da5-9a79-b41759767340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:07.725328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6edf3-4402-4da5-9a79-b41759767340 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:07.725620 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6edf3-4402-4da5-9a79-b41759767340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:07.725900 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6edf3-4402-4da5-9a79-b41759767340 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:07.726616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6edf3-4402-4da5-9a79-b41759767340 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:07.728891 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6edf3-4402-4da5-9a79-b41759767340 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:07.732668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680de95-0a4d-43b3-ae9c-a2bb4fdd669a None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:07.734039 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6680de95-0a4d-43b3-ae9c-a2bb4fdd669a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:07.753853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 979/1937] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:07 2022] POST /identity/v3/users => generated 291 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:02:07.755131 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6edf3-4402-4da5-9a79-b41759767340 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:07.768425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6edf3-4402-4da5-9a79-b41759767340 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:07.773587 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f6ea2-b709-44eb-869f-6d3475d31bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:07.774311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f6ea2-b709-44eb-869f-6d3475d31bec admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:07.774628 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f6ea2-b709-44eb-869f-6d3475d31bec admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:07.775010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8f6ea2-b709-44eb-869f-6d3475d31bec admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:07.775741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f6ea2-b709-44eb-869f-6d3475d31bec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:07.777324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8f6ea2-b709-44eb-869f-6d3475d31bec admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:07.788343 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 980/1938] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:07.792410 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f6ea2-b709-44eb-869f-6d3475d31bec admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:07.807559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6680de95-0a4d-43b3-ae9c-a2bb4fdd669a None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:07.807665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8f6ea2-b709-44eb-869f-6d3475d31bec admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:07.807665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75ffae-2c7f-4770-96b4-5559f5c3e715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:07.808185 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75ffae-2c7f-4770-96b4-5559f5c3e715 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:07.808185 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75ffae-2c7f-4770-96b4-5559f5c3e715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:07.808506 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75ffae-2c7f-4770-96b4-5559f5c3e715 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:07.809803 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75ffae-2c7f-4770-96b4-5559f5c3e715 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:07.810583 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803d4529-5f1b-4378-ab54-7b34495a4874 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:07.811278 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d4529-5f1b-4378-ab54-7b34495a4874 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:07.811494 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75ffae-2c7f-4770-96b4-5559f5c3e715 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:07.811598 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d4529-5f1b-4378-ab54-7b34495a4874 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:07.811832 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803d4529-5f1b-4378-ab54-7b34495a4874 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:07.826806 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 981/1939] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:02:07.828891 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d4529-5f1b-4378-ab54-7b34495a4874 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:07.831049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75ffae-2c7f-4770-96b4-5559f5c3e715 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:07.831479 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803d4529-5f1b-4378-ab54-7b34495a4874 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:07.843308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75ffae-2c7f-4770-96b4-5559f5c3e715 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:07.848177 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 959/1940] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:02:07.848968 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a923ab-e0ff-4da0-b06d-24c1fa3faa4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:07.850264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a923ab-e0ff-4da0-b06d-24c1fa3faa4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:07.850751 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a923ab-e0ff-4da0-b06d-24c1fa3faa4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:07.851034 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a923ab-e0ff-4da0-b06d-24c1fa3faa4d admin admin] PATH_INFO: `/v3/projects/6bd6196610c2469cb838d4f884f66c42/users/91ecaedfff12444d82813e09b142bb0f/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:07.868761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a923ab-e0ff-4da0-b06d-24c1fa3faa4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bd6196610c2469cb838d4f884f66c42, user_id=91ecaedfff12444d82813e09b142bb0f, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:07.872062 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a923ab-e0ff-4da0-b06d-24c1fa3faa4d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:07.875646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:INSERT=5,keystone:DELETE=1,keystone:SELECT=94 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:07.890995 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 982/1941] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:07 2022] PUT /identity/v3/projects/6bd6196610c2469cb838d4f884f66c42/users/91ecaedfff12444d82813e09b142bb0f/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:02:07.903670 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529d653-d0d3-4f03-b079-0f14a0ddd8cd None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:07.904604 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529d653-d0d3-4f03-b079-0f14a0ddd8cd None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:07.905470 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d529d653-d0d3-4f03-b079-0f14a0ddd8cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:07.952168 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-d529d653-d0d3-4f03-b079-0f14a0ddd8cd None None] MFA Rules not processed for user `91ecaedfff12444d82813e09b142bb0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:07.953739 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d529d653-d0d3-4f03-b079-0f14a0ddd8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:08.205102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 960/1942] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:08.218610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a923ab-e0ff-4da0-b06d-24c1fa3faa4d admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:08.220562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-75a923ab-e0ff-4da0-b06d-24c1fa3faa4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:08.291972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529d653-d0d3-4f03-b079-0f14a0ddd8cd None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:08.294398 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d529d653-d0d3-4f03-b079-0f14a0ddd8cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:08.299143 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a923ab-e0ff-4da0-b06d-24c1fa3faa4d admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:08.310018 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f70ff-bc9a-4280-8b84-cfa2a92b9953 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:08.310821 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f70ff-bc9a-4280-8b84-cfa2a92b9953 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:08.311295 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f70ff-bc9a-4280-8b84-cfa2a92b9953 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:08.311413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f70ff-bc9a-4280-8b84-cfa2a92b9953 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:08.324581 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f70ff-bc9a-4280-8b84-cfa2a92b9953 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:08.326079 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f70ff-bc9a-4280-8b84-cfa2a92b9953 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:08.335408 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:08.338741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 983/1943] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:08 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:08.359559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f70ff-bc9a-4280-8b84-cfa2a92b9953 service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:08.395280 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d529d653-d0d3-4f03-b079-0f14a0ddd8cd None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:08.404998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c50441-e1d6-4a12-8859-f87bf5d91630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:08.406907 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c50441-e1d6-4a12-8859-f87bf5d91630 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:08.407256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c50441-e1d6-4a12-8859-f87bf5d91630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:08.407606 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c50441-e1d6-4a12-8859-f87bf5d91630 admin admin] PATH_INFO: `/v3/projects/dd9386437d4e4c2fa3ef654c7a9b6c66` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:08.412228 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c50441-e1d6-4a12-8859-f87bf5d91630 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd9386437d4e4c2fa3ef654c7a9b6c66)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:08.414345 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c50441-e1d6-4a12-8859-f87bf5d91630 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:08.441721 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f70ff-bc9a-4280-8b84-cfa2a92b9953 service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:08.447448 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8b5d31-4b56-4e59-aebb-4d741a96a9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:08.448404 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b5d31-4b56-4e59-aebb-4d741a96a9d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:08.448789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b5d31-4b56-4e59-aebb-4d741a96a9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:08.449169 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8b5d31-4b56-4e59-aebb-4d741a96a9d9 admin admin] PATH_INFO: `/v3/users/91ecaedfff12444d82813e09b142bb0f` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:08.450537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b5d31-4b56-4e59-aebb-4d741a96a9d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91ecaedfff12444d82813e09b142bb0f)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:08.452811 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8b5d31-4b56-4e59-aebb-4d741a96a9d9 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:08.466222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-56c50441-e1d6-4a12-8859-f87bf5d91630 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd9386437d4e4c2fa3ef654c7a9b6c66', 'request_id': 'req-56c50441-e1d6-4a12-8859-f87bf5d91630'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:08.472765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-56c50441-e1d6-4a12-8859-f87bf5d91630 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd9386437d4e4c2fa3ef654c7a9b6c66', 'request_id': 'req-56c50441-e1d6-4a12-8859-f87bf5d91630'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:08.481541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-56c50441-e1d6-4a12-8859-f87bf5d91630 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd9386437d4e4c2fa3ef654c7a9b6c66', 'request_id': 'req-56c50441-e1d6-4a12-8859-f87bf5d91630'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:08.499131 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-56c50441-e1d6-4a12-8859-f87bf5d91630 admin admin] The token cache is being invalidate because project dd9386437d4e4c2fa3ef654c7a9b6c66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:08.499429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-56c50441-e1d6-4a12-8859-f87bf5d91630 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:08.500312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-56c50441-e1d6-4a12-8859-f87bf5d91630 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:08.502114 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 961/1944] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:08 2022] DELETE /identity/v3/projects/dd9386437d4e4c2fa3ef654c7a9b6c66 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:08.505911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c50441-e1d6-4a12-8859-f87bf5d91630 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:08.507667 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-56c50441-e1d6-4a12-8859-f87bf5d91630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:08.515895 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1b8b5d31-4b56-4e59-aebb-4d741a96a9d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91ecaedfff12444d82813e09b142bb0f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:08.523589 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1b8b5d31-4b56-4e59-aebb-4d741a96a9d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91ecaedfff12444d82813e09b142bb0f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:08.524381 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1b8b5d31-4b56-4e59-aebb-4d741a96a9d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91ecaedfff12444d82813e09b142bb0f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:08.538636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1b8b5d31-4b56-4e59-aebb-4d741a96a9d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91ecaedfff12444d82813e09b142bb0f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:08.539918 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1b8b5d31-4b56-4e59-aebb-4d741a96a9d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91ecaedfff12444d82813e09b142bb0f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:08.555006 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 984/1945] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:08 2022] DELETE /identity/v3/users/91ecaedfff12444d82813e09b142bb0f => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:02:08.559550 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866624e-b639-4068-a5db-b223d164bd86 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:08.559873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866624e-b639-4068-a5db-b223d164bd86 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:08.560174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3866624e-b639-4068-a5db-b223d164bd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:08.577109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-3866624e-b639-4068-a5db-b223d164bd86 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:08.578238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-3866624e-b639-4068-a5db-b223d164bd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:08.585932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c50441-e1d6-4a12-8859-f87bf5d91630 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:08.592156 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:08.642382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:08.649966 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf583edd-074a-4a59-8c09-19eb6f2de0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:08.650923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf583edd-074a-4a59-8c09-19eb6f2de0ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:08.652129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf583edd-074a-4a59-8c09-19eb6f2de0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:08.653605 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf583edd-074a-4a59-8c09-19eb6f2de0ba admin admin] PATH_INFO: `/v3/users/a6633781fb52435f9ce99ad7912617dd` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:08.655301 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf583edd-074a-4a59-8c09-19eb6f2de0ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6633781fb52435f9ce99ad7912617dd)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:08.656688 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 985/1946] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:08.660973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf583edd-074a-4a59-8c09-19eb6f2de0ba admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:08.667135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866624e-b639-4068-a5db-b223d164bd86 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:08.669057 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-3866624e-b639-4068-a5db-b223d164bd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:08.727140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cf583edd-074a-4a59-8c09-19eb6f2de0ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6633781fb52435f9ce99ad7912617dd'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:08.728294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cf583edd-074a-4a59-8c09-19eb6f2de0ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6633781fb52435f9ce99ad7912617dd'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:08.740505 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cf583edd-074a-4a59-8c09-19eb6f2de0ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6633781fb52435f9ce99ad7912617dd'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:08.741943 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cf583edd-074a-4a59-8c09-19eb6f2de0ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6633781fb52435f9ce99ad7912617dd'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:08.756634 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cf583edd-074a-4a59-8c09-19eb6f2de0ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6633781fb52435f9ce99ad7912617dd'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:08.761465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3866624e-b639-4068-a5db-b223d164bd86 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:08.765860 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e63e2f-e700-4eb6-8e03-979517e90d42 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:08.766515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e63e2f-e700-4eb6-8e03-979517e90d42 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:08.766833 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e63e2f-e700-4eb6-8e03-979517e90d42 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:08.767114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e63e2f-e700-4eb6-8e03-979517e90d42 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:08.769138 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-70e63e2f-e700-4eb6-8e03-979517e90d42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:08.770483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 962/1947] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:08 2022] DELETE /identity/v3/users/a6633781fb52435f9ce99ad7912617dd => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:02:08.785006 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e63e2f-e700-4eb6-8e03-979517e90d42 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:08.786436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e63e2f-e700-4eb6-8e03-979517e90d42 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:08.797195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 986/1948] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:09.237478 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf583edd-074a-4a59-8c09-19eb6f2de0ba admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:09.239178 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-cf583edd-074a-4a59-8c09-19eb6f2de0ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:09.270258 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e63e2f-e700-4eb6-8e03-979517e90d42 service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:09.272357 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-70e63e2f-e700-4eb6-8e03-979517e90d42 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:09.290215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e63e2f-e700-4eb6-8e03-979517e90d42 service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:09.339504 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf583edd-074a-4a59-8c09-19eb6f2de0ba admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:09.345867 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dd4cf9-a17e-49b5-a4d5-a9a69be6a409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:09.346933 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd4cf9-a17e-49b5-a4d5-a9a69be6a409 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:09.347322 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd4cf9-a17e-49b5-a4d5-a9a69be6a409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:09.347702 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dd4cf9-a17e-49b5-a4d5-a9a69be6a409 admin admin] PATH_INFO: `/v3/projects/4570a205938b40029454385e7940d5e8` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:09.349005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd4cf9-a17e-49b5-a4d5-a9a69be6a409 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4570a205938b40029454385e7940d5e8)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:09.351084 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21dd4cf9-a17e-49b5-a4d5-a9a69be6a409 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:09.370775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dc31d1-3dba-4273-9f31-91c252165734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:09.372119 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc31d1-3dba-4273-9f31-91c252165734 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:09.372560 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc31d1-3dba-4273-9f31-91c252165734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:09.373300 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dc31d1-3dba-4273-9f31-91c252165734 admin admin] PATH_INFO: `/v3/projects/6bd6196610c2469cb838d4f884f66c42` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:09.375687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc31d1-3dba-4273-9f31-91c252165734 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bd6196610c2469cb838d4f884f66c42)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:09.377952 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dc31d1-3dba-4273-9f31-91c252165734 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:09.414418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-21dd4cf9-a17e-49b5-a4d5-a9a69be6a409 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4570a205938b40029454385e7940d5e8', 'request_id': 'req-21dd4cf9-a17e-49b5-a4d5-a9a69be6a409'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:09.421449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-21dd4cf9-a17e-49b5-a4d5-a9a69be6a409 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4570a205938b40029454385e7940d5e8', 'request_id': 'req-21dd4cf9-a17e-49b5-a4d5-a9a69be6a409'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:09.428122 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-21dd4cf9-a17e-49b5-a4d5-a9a69be6a409 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4570a205938b40029454385e7940d5e8', 'request_id': 'req-21dd4cf9-a17e-49b5-a4d5-a9a69be6a409'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:09.441153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-21dd4cf9-a17e-49b5-a4d5-a9a69be6a409 admin admin] The token cache is being invalidate because project 4570a205938b40029454385e7940d5e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:09.441315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-79dc31d1-3dba-4273-9f31-91c252165734 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bd6196610c2469cb838d4f884f66c42', 'request_id': 'req-79dc31d1-3dba-4273-9f31-91c252165734'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:09.441433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-21dd4cf9-a17e-49b5-a4d5-a9a69be6a409 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:09.442002 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-21dd4cf9-a17e-49b5-a4d5-a9a69be6a409 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:09.443577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 963/1949] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:09 2022] DELETE /identity/v3/projects/4570a205938b40029454385e7940d5e8 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:09.447798 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-79dc31d1-3dba-4273-9f31-91c252165734 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bd6196610c2469cb838d4f884f66c42', 'request_id': 'req-79dc31d1-3dba-4273-9f31-91c252165734'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:09.448208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec4de6-8eaa-4157-9e02-165183b2db56 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:09.448504 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec4de6-8eaa-4157-9e02-165183b2db56 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:09.448918 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ec4de6-8eaa-4157-9e02-165183b2db56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:09.452513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:09.453782 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-79dc31d1-3dba-4273-9f31-91c252165734 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bd6196610c2469cb838d4f884f66c42', 'request_id': 'req-79dc31d1-3dba-4273-9f31-91c252165734'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:09.463502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:09.469452 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-79dc31d1-3dba-4273-9f31-91c252165734 admin admin] The token cache is being invalidate because project 6bd6196610c2469cb838d4f884f66c42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:09.469776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-c1ec4de6-8eaa-4157-9e02-165183b2db56 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:09.469881 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-79dc31d1-3dba-4273-9f31-91c252165734 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:09.470556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-79dc31d1-3dba-4273-9f31-91c252165734 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:09.472095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c1ec4de6-8eaa-4157-9e02-165183b2db56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:09.472553 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 987/1950] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:09 2022] DELETE /identity/v3/projects/6bd6196610c2469cb838d4f884f66c42 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:02:09.478973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80dd8a-1c4e-4288-af97-359cf12f7275 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:09.479350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80dd8a-1c4e-4288-af97-359cf12f7275 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:09.479636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80dd8a-1c4e-4288-af97-359cf12f7275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:09.498065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-0e80dd8a-1c4e-4288-af97-359cf12f7275 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:09.499277 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-0e80dd8a-1c4e-4288-af97-359cf12f7275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:09.533261 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 964/1951] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:09.542495 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d60f18-b469-461c-9b48-2fb0d3cb2b29 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:09.542852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d60f18-b469-461c-9b48-2fb0d3cb2b29 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:09.543731 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d60f18-b469-461c-9b48-2fb0d3cb2b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:09.559479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 988/1952] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:09.561501 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-46d60f18-b469-461c-9b48-2fb0d3cb2b29 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:09.562796 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-46d60f18-b469-461c-9b48-2fb0d3cb2b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:09.567144 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86bc97-f517-4ecc-9d14-01b765ea2af0 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:09.567447 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86bc97-f517-4ecc-9d14-01b765ea2af0 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:09.567742 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86bc97-f517-4ecc-9d14-01b765ea2af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:09.570950 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 965/1953] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:09.574858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d60f18-b469-461c-9b48-2fb0d3cb2b29 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:09.583764 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d60f18-b469-461c-9b48-2fb0d3cb2b29 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:09.585493 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-5c86bc97-f517-4ecc-9d14-01b765ea2af0 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:09.586727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5c86bc97-f517-4ecc-9d14-01b765ea2af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:09.588427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89471f9d-5a23-4413-96e5-150be37c8462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:09.589069 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89471f9d-5a23-4413-96e5-150be37c8462 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:09.589357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89471f9d-5a23-4413-96e5-150be37c8462 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:09.589641 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89471f9d-5a23-4413-96e5-150be37c8462 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:09.590930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89471f9d-5a23-4413-96e5-150be37c8462 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:02:09.591266 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89471f9d-5a23-4413-96e5-150be37c8462 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:09.592849 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89471f9d-5a23-4413-96e5-150be37c8462 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:09.595797 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 989/1954] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:09.599547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86bc97-f517-4ecc-9d14-01b765ea2af0 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:09.604698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 966/1955] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 23:02:09.609338 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89471f9d-5a23-4413-96e5-150be37c8462 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:09.613074 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86bc97-f517-4ecc-9d14-01b765ea2af0 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:09.617787 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9655a-9f1f-43d8-beaa-e8d215d17501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:09.618472 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9655a-9f1f-43d8-beaa-e8d215d17501 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:09.618792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89471f9d-5a23-4413-96e5-150be37c8462 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:09.618892 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9655a-9f1f-43d8-beaa-e8d215d17501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:09.619109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb9655a-9f1f-43d8-beaa-e8d215d17501 admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:09.619835 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9655a-9f1f-43d8-beaa-e8d215d17501 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:02:09.620146 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9655a-9f1f-43d8-beaa-e8d215d17501 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:09.621767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb9655a-9f1f-43d8-beaa-e8d215d17501 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:09.623470 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6e1f3-5c99-4e38-9b8c-000019ad5603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:09.624101 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6e1f3-5c99-4e38-9b8c-000019ad5603 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:09.624382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6e1f3-5c99-4e38-9b8c-000019ad5603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:09.624675 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c6e1f3-5c99-4e38-9b8c-000019ad5603 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:09.625345 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c6e1f3-5c99-4e38-9b8c-000019ad5603 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:09.627260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c6e1f3-5c99-4e38-9b8c-000019ad5603 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:09.637848 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 990/1956] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:02:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 23:02:09.642922 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9655a-9f1f-43d8-beaa-e8d215d17501 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:09.655236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb9655a-9f1f-43d8-beaa-e8d215d17501 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:09.660535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82effe5-20a2-4d2f-bf05-c932fa2c8176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:09.661211 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82effe5-20a2-4d2f-bf05-c932fa2c8176 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:09.661501 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82effe5-20a2-4d2f-bf05-c932fa2c8176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:09.661786 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82effe5-20a2-4d2f-bf05-c932fa2c8176 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:09.662493 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82effe5-20a2-4d2f-bf05-c932fa2c8176 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:09.664465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82effe5-20a2-4d2f-bf05-c932fa2c8176 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:09.666279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 967/1957] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:09 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:02:09.670313 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6e1f3-5c99-4e38-9b8c-000019ad5603 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:09.696102 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 991/1958] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:09 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:02:09.697338 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82effe5-20a2-4d2f-bf05-c932fa2c8176 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:09.739549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c6e1f3-5c99-4e38-9b8c-000019ad5603 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:09.760165 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82effe5-20a2-4d2f-bf05-c932fa2c8176 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:09.766783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0600764-4f10-4021-a333-01162c1f8aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:09.767588 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0600764-4f10-4021-a333-01162c1f8aae admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:09.768357 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0600764-4f10-4021-a333-01162c1f8aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:09.768357 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0600764-4f10-4021-a333-01162c1f8aae admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:09.769124 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0600764-4f10-4021-a333-01162c1f8aae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:09.771162 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0600764-4f10-4021-a333-01162c1f8aae admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:09.794295 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 992/1959] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:09 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:02:09.798017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0600764-4f10-4021-a333-01162c1f8aae admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:09.807684 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0600764-4f10-4021-a333-01162c1f8aae admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:09.809499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2b6b8-42b9-439c-9e35-fe78a74fae31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:09.810187 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2b6b8-42b9-439c-9e35-fe78a74fae31 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:09.810499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2b6b8-42b9-439c-9e35-fe78a74fae31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:09.810814 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2b6b8-42b9-439c-9e35-fe78a74fae31 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:09.811548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2b6b8-42b9-439c-9e35-fe78a74fae31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:09.812711 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d5e68-bc14-4b60-8328-f1d251ba4934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:09.813365 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d5e68-bc14-4b60-8328-f1d251ba4934 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:09.813474 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2b6b8-42b9-439c-9e35-fe78a74fae31 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:09.813654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d5e68-bc14-4b60-8328-f1d251ba4934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:09.813935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0d5e68-bc14-4b60-8328-f1d251ba4934 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:09.814691 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d5e68-bc14-4b60-8328-f1d251ba4934 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:09.816305 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0d5e68-bc14-4b60-8328-f1d251ba4934 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:09.827814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 993/1960] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:09.831526 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d5e68-bc14-4b60-8328-f1d251ba4934 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:09.837048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 968/1961] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:09 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:02:09.841123 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2b6b8-42b9-439c-9e35-fe78a74fae31 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:09.841484 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0d5e68-bc14-4b60-8328-f1d251ba4934 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:09.846266 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:09.846470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64898fa1-9cf2-4a69-95cd-a92fd1700ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:09.847299 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64898fa1-9cf2-4a69-95cd-a92fd1700ea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:09.847921 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64898fa1-9cf2-4a69-95cd-a92fd1700ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:09.848248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64898fa1-9cf2-4a69-95cd-a92fd1700ea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:09.849041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64898fa1-9cf2-4a69-95cd-a92fd1700ea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:09.850812 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64898fa1-9cf2-4a69-95cd-a92fd1700ea2 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:09.853314 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2b6b8-42b9-439c-9e35-fe78a74fae31 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:09.859552 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7deb4b-7971-40b7-a837-2890a5b909a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:09.860200 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7deb4b-7971-40b7-a837-2890a5b909a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:09.860489 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7deb4b-7971-40b7-a837-2890a5b909a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:09.860769 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7deb4b-7971-40b7-a837-2890a5b909a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:09.861492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7deb4b-7971-40b7-a837-2890a5b909a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:09.862618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 994/1962] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:02:09.863205 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7deb4b-7971-40b7-a837-2890a5b909a2 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:09.866794 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64898fa1-9cf2-4a69-95cd-a92fd1700ea2 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:09.875509 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 969/1963] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:02:09.876835 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64898fa1-9cf2-4a69-95cd-a92fd1700ea2 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:09.879362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7deb4b-7971-40b7-a837-2890a5b909a2 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:09.881520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92692cfe-bea3-4a88-8b27-5af0af4b10e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:09.882398 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92692cfe-bea3-4a88-8b27-5af0af4b10e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:09.882720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92692cfe-bea3-4a88-8b27-5af0af4b10e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:09.883002 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92692cfe-bea3-4a88-8b27-5af0af4b10e9 admin admin] PATH_INFO: `/v3/projects/c90a3d32fe204da7a969528722141e75/users/0ee1c54733f648ada66a143ce7516375/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:09.889994 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7deb4b-7971-40b7-a837-2890a5b909a2 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:09.895128 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268a82c-af26-4006-9fe5-61e3f8d110a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:09.895824 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268a82c-af26-4006-9fe5-61e3f8d110a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:09.896117 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268a82c-af26-4006-9fe5-61e3f8d110a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:09.896404 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e268a82c-af26-4006-9fe5-61e3f8d110a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:09.897254 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268a82c-af26-4006-9fe5-61e3f8d110a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:09.898839 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e268a82c-af26-4006-9fe5-61e3f8d110a6 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:09.905497 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92692cfe-bea3-4a88-8b27-5af0af4b10e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c90a3d32fe204da7a969528722141e75, user_id=0ee1c54733f648ada66a143ce7516375, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:09.908537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:09.909320 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92692cfe-bea3-4a88-8b27-5af0af4b10e9 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:09.911054 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 970/1964] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:02:09.915355 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268a82c-af26-4006-9fe5-61e3f8d110a6 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:09.927800 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e268a82c-af26-4006-9fe5-61e3f8d110a6 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:09.929073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 995/1965] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:09 2022] PUT /identity/v3/projects/c90a3d32fe204da7a969528722141e75/users/0ee1c54733f648ada66a143ce7516375/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:02:09.940282 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21ea5e-ae95-4021-8c85-c992a04dd4e5 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:09.940605 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21ea5e-ae95-4021-8c85-c992a04dd4e5 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:09.940973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba21ea5e-ae95-4021-8c85-c992a04dd4e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:09.969857 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-ba21ea5e-ae95-4021-8c85-c992a04dd4e5 None None] MFA Rules not processed for user `0ee1c54733f648ada66a143ce7516375`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:09.971108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ba21ea5e-ae95-4021-8c85-c992a04dd4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:10.002098 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb6bc6c-5d55-47a2-b527-585445ce2089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:10.003035 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6bc6c-5d55-47a2-b527-585445ce2089 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:10.003324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6bc6c-5d55-47a2-b527-585445ce2089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:10.003605 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb6bc6c-5d55-47a2-b527-585445ce2089 admin admin] PATH_INFO: `/v3/projects/66f00e07b07d425fa2012d46b7b59649/users/4598214a0fce4a18aaeed3270f990324/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:10.031682 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6bc6c-5d55-47a2-b527-585445ce2089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66f00e07b07d425fa2012d46b7b59649, user_id=4598214a0fce4a18aaeed3270f990324, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:10.035132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb6bc6c-5d55-47a2-b527-585445ce2089 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:10.045867 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 996/1966] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:10.052071 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 971/1967] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:09 2022] PUT /identity/v3/projects/66f00e07b07d425fa2012d46b7b59649/users/4598214a0fce4a18aaeed3270f990324/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:02:10.061453 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21ea5e-ae95-4021-8c85-c992a04dd4e5 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:10.063356 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ba21ea5e-ae95-4021-8c85-c992a04dd4e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:10.063769 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2e197-5297-47b0-bbe0-cac38ceb5cc4 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:10.064261 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2e197-5297-47b0-bbe0-cac38ceb5cc4 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:10.064569 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c2e197-5297-47b0-bbe0-cac38ceb5cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:10.098676 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-91c2e197-5297-47b0-bbe0-cac38ceb5cc4 None None] MFA Rules not processed for user `4598214a0fce4a18aaeed3270f990324`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:10.099891 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-91c2e197-5297-47b0-bbe0-cac38ceb5cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:10.127178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba21ea5e-ae95-4021-8c85-c992a04dd4e5 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:10.171380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 972/1968] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:10.175923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2e197-5297-47b0-bbe0-cac38ceb5cc4 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:10.179063 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8bacf-fec8-4bc7-9638-fcf0ec5e449f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:10.179788 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8bacf-fec8-4bc7-9638-fcf0ec5e449f service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:10.180123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8bacf-fec8-4bc7-9638-fcf0ec5e449f service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:10.180410 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de8bacf-fec8-4bc7-9638-fcf0ec5e449f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:10.233196 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:10.242907 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:10.255520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de8bacf-fec8-4bc7-9638-fcf0ec5e449f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:10.258159 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de8bacf-fec8-4bc7-9638-fcf0ec5e449f service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:10.262343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c2e197-5297-47b0-bbe0-cac38ceb5cc4 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:10.270020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6873c6-ad79-4ee8-b119-1b50924428c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:10.270754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6873c6-ad79-4ee8-b119-1b50924428c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:10.271046 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6873c6-ad79-4ee8-b119-1b50924428c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:10.271270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 997/1969] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:10.271741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6873c6-ad79-4ee8-b119-1b50924428c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:10.272011 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6873c6-ad79-4ee8-b119-1b50924428c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:10.274044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6873c6-ad79-4ee8-b119-1b50924428c0 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:10.301908 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 973/1970] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:10 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:02:10.306033 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8bacf-fec8-4bc7-9638-fcf0ec5e449f service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:10.318361 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de8bacf-fec8-4bc7-9638-fcf0ec5e449f service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:10.376649 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda29bb8-559a-4faf-9e97-3a9c05b52cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:10.377330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda29bb8-559a-4faf-9e97-3a9c05b52cdf admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:10.377619 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda29bb8-559a-4faf-9e97-3a9c05b52cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:10.377902 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda29bb8-559a-4faf-9e97-3a9c05b52cdf admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:10.378677 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda29bb8-559a-4faf-9e97-3a9c05b52cdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:10.380630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda29bb8-559a-4faf-9e97-3a9c05b52cdf admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:10.401649 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 998/1971] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:10 2022] POST /identity/v3/users => generated 299 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:02:10.405882 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6873c6-ad79-4ee8-b119-1b50924428c0 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:10.416895 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6873c6-ad79-4ee8-b119-1b50924428c0 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:10.423264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e69202-d964-4d51-b2e0-0870f919896c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:10.423947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e69202-d964-4d51-b2e0-0870f919896c admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:10.424242 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e69202-d964-4d51-b2e0-0870f919896c admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:10.424527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e69202-d964-4d51-b2e0-0870f919896c admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:10.425263 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e69202-d964-4d51-b2e0-0870f919896c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:10.427043 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e69202-d964-4d51-b2e0-0870f919896c admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:10.439494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 974/1972] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:10.443855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda29bb8-559a-4faf-9e97-3a9c05b52cdf admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:10.454172 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda29bb8-559a-4faf-9e97-3a9c05b52cdf admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:10.459447 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581719a-0cdf-4021-9ae6-9d59674763fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:10.460397 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581719a-0cdf-4021-9ae6-9d59674763fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:10.460831 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581719a-0cdf-4021-9ae6-9d59674763fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:10.461120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1581719a-0cdf-4021-9ae6-9d59674763fa admin admin] PATH_INFO: `/v3/projects/feb78b24bf9b40f0a89c24b553aefbf2/users/96098724668e4a78be58aa7509a0462c/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:10.477164 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581719a-0cdf-4021-9ae6-9d59674763fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=feb78b24bf9b40f0a89c24b553aefbf2, user_id=96098724668e4a78be58aa7509a0462c, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:10.480824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1581719a-0cdf-4021-9ae6-9d59674763fa admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:10.497175 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 999/1973] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:10 2022] PUT /identity/v3/projects/feb78b24bf9b40f0a89c24b553aefbf2/users/96098724668e4a78be58aa7509a0462c/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:10.501386 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e69202-d964-4d51-b2e0-0870f919896c admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:10.512945 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e69202-d964-4d51-b2e0-0870f919896c admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:10.606255 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebde2e56-33fd-4be9-b740-60c696eb0c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:10.607165 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde2e56-33fd-4be9-b740-60c696eb0c1b admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:10.607461 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde2e56-33fd-4be9-b740-60c696eb0c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:10.607740 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebde2e56-33fd-4be9-b740-60c696eb0c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:10.608479 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde2e56-33fd-4be9-b740-60c696eb0c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:10.610090 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebde2e56-33fd-4be9-b740-60c696eb0c1b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:10.624390 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 975/1974] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:10 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:10.625404 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581719a-0cdf-4021-9ae6-9d59674763fa admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:10.636064 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1581719a-0cdf-4021-9ae6-9d59674763fa admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:10.641294 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361b1c3-c439-4054-9935-8c8f359b553e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:10.642198 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361b1c3-c439-4054-9935-8c8f359b553e admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:10.642487 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361b1c3-c439-4054-9935-8c8f359b553e admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:10.642810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3361b1c3-c439-4054-9935-8c8f359b553e admin admin] PATH_INFO: `/v3/projects/feb78b24bf9b40f0a89c24b553aefbf2/users/96098724668e4a78be58aa7509a0462c/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:10.644800 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3361b1c3-c439-4054-9935-8c8f359b553e admin admin] RBAC: Authorizing `identity:create_grant(project_id=feb78b24bf9b40f0a89c24b553aefbf2, user_id=96098724668e4a78be58aa7509a0462c, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:10.647786 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3361b1c3-c439-4054-9935-8c8f359b553e admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:10.659480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1000/1975] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:10 2022] PUT /identity/v3/projects/feb78b24bf9b40f0a89c24b553aefbf2/users/96098724668e4a78be58aa7509a0462c/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:10.670673 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc288962-035c-4908-919e-91dd651f8dd9 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:10.671127 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc288962-035c-4908-919e-91dd651f8dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:10.671243 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc288962-035c-4908-919e-91dd651f8dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:10.698178 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-cc288962-035c-4908-919e-91dd651f8dd9 None None] MFA Rules not processed for user `96098724668e4a78be58aa7509a0462c`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:10.699999 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-cc288962-035c-4908-919e-91dd651f8dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:10.769826 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 976/1976] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:10 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:10.781403 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361b1c3-c439-4054-9935-8c8f359b553e admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:10.835914 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3361b1c3-c439-4054-9935-8c8f359b553e admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:10.840537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16e071-a7e8-4f77-9c6e-8cb6ff9c15c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:10.841191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16e071-a7e8-4f77-9c6e-8cb6ff9c15c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:10.841474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16e071-a7e8-4f77-9c6e-8cb6ff9c15c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:10.841747 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16e071-a7e8-4f77-9c6e-8cb6ff9c15c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:10.851094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16e071-a7e8-4f77-9c6e-8cb6ff9c15c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:10.852751 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16e071-a7e8-4f77-9c6e-8cb6ff9c15c8 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:10.862673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1001/1977] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:10 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:11.439809 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc288962-035c-4908-919e-91dd651f8dd9 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:11.489353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:11.503320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc288962-035c-4908-919e-91dd651f8dd9 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:11.507729 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82f5671-2deb-4acf-8a6e-a938f6b868a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:11.508601 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f5671-2deb-4acf-8a6e-a938f6b868a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:11.508890 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f5671-2deb-4acf-8a6e-a938f6b868a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:11.509164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82f5671-2deb-4acf-8a6e-a938f6b868a9 admin admin] PATH_INFO: `/v3/users/4598214a0fce4a18aaeed3270f990324` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:11.510353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f5671-2deb-4acf-8a6e-a938f6b868a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4598214a0fce4a18aaeed3270f990324)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:11.512824 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82f5671-2deb-4acf-8a6e-a938f6b868a9 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:11.569953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-e82f5671-2deb-4acf-8a6e-a938f6b868a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4598214a0fce4a18aaeed3270f990324'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:11.570622 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-e82f5671-2deb-4acf-8a6e-a938f6b868a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4598214a0fce4a18aaeed3270f990324'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:11.578017 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-e82f5671-2deb-4acf-8a6e-a938f6b868a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4598214a0fce4a18aaeed3270f990324'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:11.578679 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-e82f5671-2deb-4acf-8a6e-a938f6b868a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4598214a0fce4a18aaeed3270f990324'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:11.590838 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-e82f5671-2deb-4acf-8a6e-a938f6b868a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4598214a0fce4a18aaeed3270f990324'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:11.603639 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 977/1978] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:11 2022] DELETE /identity/v3/users/4598214a0fce4a18aaeed3270f990324 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:11.608499 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7be3fb-a483-4520-839e-d3ec4555b365 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:11.609021 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7be3fb-a483-4520-839e-d3ec4555b365 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:11.609099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7be3fb-a483-4520-839e-d3ec4555b365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:11.629282 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-df7be3fb-a483-4520-839e-d3ec4555b365 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:11.629538 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-df7be3fb-a483-4520-839e-d3ec4555b365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:11.677515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:11.690809 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9fc2b-6d29-4d0a-a66a-9ab2f09ae0a3 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:11.691336 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9fc2b-6d29-4d0a-a66a-9ab2f09ae0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:11.691573 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9fc2b-6d29-4d0a-a66a-9ab2f09ae0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:11.697366 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1002/1979] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:11.707684 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7be3fb-a483-4520-839e-d3ec4555b365 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:11.709284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-df7be3fb-a483-4520-839e-d3ec4555b365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:11.710904 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-baf9fc2b-6d29-4d0a-a66a-9ab2f09ae0a3 None None] MFA Rules not processed for user `0ee1c54733f648ada66a143ce7516375`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:11.712196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-baf9fc2b-6d29-4d0a-a66a-9ab2f09ae0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:11.774008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 978/1980] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:11.774157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7be3fb-a483-4520-839e-d3ec4555b365 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:11.780317 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8710f-c251-4493-bf43-7472aef69eba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:11.781045 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8710f-c251-4493-bf43-7472aef69eba service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:11.781420 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8710f-c251-4493-bf43-7472aef69eba service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:11.781758 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd8710f-c251-4493-bf43-7472aef69eba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:11.791801 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8710f-c251-4493-bf43-7472aef69eba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:11.793304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd8710f-c251-4493-bf43-7472aef69eba service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:11.802780 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1003/1981] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:02:12.101974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9fc2b-6d29-4d0a-a66a-9ab2f09ae0a3 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:12.103975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-baf9fc2b-6d29-4d0a-a66a-9ab2f09ae0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:12.136778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9fc2b-6d29-4d0a-a66a-9ab2f09ae0a3 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:12.142062 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40108f9e-e28c-4e2e-992a-0515fa116dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:12.143003 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40108f9e-e28c-4e2e-992a-0515fa116dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:12.143692 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40108f9e-e28c-4e2e-992a-0515fa116dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:12.143979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40108f9e-e28c-4e2e-992a-0515fa116dd2 admin admin] PATH_INFO: `/v3/projects/66f00e07b07d425fa2012d46b7b59649` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:12.145319 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40108f9e-e28c-4e2e-992a-0515fa116dd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66f00e07b07d425fa2012d46b7b59649)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:12.147743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40108f9e-e28c-4e2e-992a-0515fa116dd2 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:12.198008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-40108f9e-e28c-4e2e-992a-0515fa116dd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66f00e07b07d425fa2012d46b7b59649', 'request_id': 'req-40108f9e-e28c-4e2e-992a-0515fa116dd2'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.204490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-40108f9e-e28c-4e2e-992a-0515fa116dd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66f00e07b07d425fa2012d46b7b59649', 'request_id': 'req-40108f9e-e28c-4e2e-992a-0515fa116dd2'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.211019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-40108f9e-e28c-4e2e-992a-0515fa116dd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66f00e07b07d425fa2012d46b7b59649', 'request_id': 'req-40108f9e-e28c-4e2e-992a-0515fa116dd2'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.214872 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:12.229981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-40108f9e-e28c-4e2e-992a-0515fa116dd2 admin admin] The token cache is being invalidate because project 66f00e07b07d425fa2012d46b7b59649 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:12.230893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-40108f9e-e28c-4e2e-992a-0515fa116dd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.231601 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-40108f9e-e28c-4e2e-992a-0515fa116dd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.233300 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 979/1982] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:12 2022] DELETE /identity/v3/projects/66f00e07b07d425fa2012d46b7b59649 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:12.237641 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8710f-c251-4493-bf43-7472aef69eba service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:12.237807 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40108f9e-e28c-4e2e-992a-0515fa116dd2 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:12.239957 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-40108f9e-e28c-4e2e-992a-0515fa116dd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:12.250407 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd8710f-c251-4493-bf43-7472aef69eba service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:12.344541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40108f9e-e28c-4e2e-992a-0515fa116dd2 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:12.347026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f251b3b-5e86-4b24-b999-b229202c1ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:12.347964 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f251b3b-5e86-4b24-b999-b229202c1ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:12.354793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f251b3b-5e86-4b24-b999-b229202c1ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:12.354793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f251b3b-5e86-4b24-b999-b229202c1ed8 admin admin] PATH_INFO: `/v3/users/96098724668e4a78be58aa7509a0462c` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:12.354793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f251b3b-5e86-4b24-b999-b229202c1ed8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96098724668e4a78be58aa7509a0462c)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:12.354793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f251b3b-5e86-4b24-b999-b229202c1ed8 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:12.365048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20897b-e982-4502-b015-faf596143ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:12.365968 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20897b-e982-4502-b015-faf596143ac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:12.366320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20897b-e982-4502-b015-faf596143ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:12.366693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f20897b-e982-4502-b015-faf596143ac6 admin admin] PATH_INFO: `/v3/users/e5770101917a49269e6850a2003071c5` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:12.368318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f20897b-e982-4502-b015-faf596143ac6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5770101917a49269e6850a2003071c5)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:12.372317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f20897b-e982-4502-b015-faf596143ac6 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:12.417528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9f251b3b-5e86-4b24-b999-b229202c1ed8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96098724668e4a78be58aa7509a0462c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.425568 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9f251b3b-5e86-4b24-b999-b229202c1ed8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96098724668e4a78be58aa7509a0462c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.426283 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9f251b3b-5e86-4b24-b999-b229202c1ed8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96098724668e4a78be58aa7509a0462c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.435849 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-4f20897b-e982-4502-b015-faf596143ac6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5770101917a49269e6850a2003071c5'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.436955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-4f20897b-e982-4502-b015-faf596143ac6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5770101917a49269e6850a2003071c5'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.441211 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9f251b3b-5e86-4b24-b999-b229202c1ed8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96098724668e4a78be58aa7509a0462c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.442508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9f251b3b-5e86-4b24-b999-b229202c1ed8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96098724668e4a78be58aa7509a0462c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.445073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-4f20897b-e982-4502-b015-faf596143ac6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5770101917a49269e6850a2003071c5'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.446537 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-4f20897b-e982-4502-b015-faf596143ac6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5770101917a49269e6850a2003071c5'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.458260 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1004/1983] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:12 2022] DELETE /identity/v3/users/96098724668e4a78be58aa7509a0462c => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:12.467250 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-4f20897b-e982-4502-b015-faf596143ac6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5770101917a49269e6850a2003071c5'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.482105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 980/1984] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:12 2022] DELETE /identity/v3/users/e5770101917a49269e6850a2003071c5 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:12.486957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfad64-aeb4-4ee0-b323-98b2d202eef0 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:12.487480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfad64-aeb4-4ee0-b323-98b2d202eef0 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:12.487767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfad64-aeb4-4ee0-b323-98b2d202eef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:12.507892 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-39bfad64-aeb4-4ee0-b323-98b2d202eef0 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:12.509101 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-39bfad64-aeb4-4ee0-b323-98b2d202eef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:12.514503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:12.578682 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1005/1985] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:12.591830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20897b-e982-4502-b015-faf596143ac6 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:12.593409 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-4f20897b-e982-4502-b015-faf596143ac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:12.663491 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f20897b-e982-4502-b015-faf596143ac6 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:12.668525 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b2ac8-4ffd-4b10-920f-19d5a63f8fac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:12.669196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b2ac8-4ffd-4b10-920f-19d5a63f8fac service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:12.669560 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b2ac8-4ffd-4b10-920f-19d5a63f8fac service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:12.670268 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b2ac8-4ffd-4b10-920f-19d5a63f8fac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:12.681837 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b2ac8-4ffd-4b10-920f-19d5a63f8fac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:12.684005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b2ac8-4ffd-4b10-920f-19d5a63f8fac service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:12.695811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 981/1986] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:12.863924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bfad64-aeb4-4ee0-b323-98b2d202eef0 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:12.865714 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-39bfad64-aeb4-4ee0-b323-98b2d202eef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:12.878486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bfad64-aeb4-4ee0-b323-98b2d202eef0 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:12.886946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1373cc75-d0aa-47d9-94ea-860f5fc2008c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:12.888262 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373cc75-d0aa-47d9-94ea-860f5fc2008c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:12.888461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373cc75-d0aa-47d9-94ea-860f5fc2008c admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:12.890733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1373cc75-d0aa-47d9-94ea-860f5fc2008c admin admin] PATH_INFO: `/v3/projects/feb78b24bf9b40f0a89c24b553aefbf2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:12.891598 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373cc75-d0aa-47d9-94ea-860f5fc2008c admin admin] RBAC: Authorizing `identity:delete_project(project_id=feb78b24bf9b40f0a89c24b553aefbf2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:12.895814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1373cc75-d0aa-47d9-94ea-860f5fc2008c admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:12.963420 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1373cc75-d0aa-47d9-94ea-860f5fc2008c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feb78b24bf9b40f0a89c24b553aefbf2', 'request_id': 'req-1373cc75-d0aa-47d9-94ea-860f5fc2008c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.972726 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1373cc75-d0aa-47d9-94ea-860f5fc2008c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feb78b24bf9b40f0a89c24b553aefbf2', 'request_id': 'req-1373cc75-d0aa-47d9-94ea-860f5fc2008c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.980938 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1373cc75-d0aa-47d9-94ea-860f5fc2008c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feb78b24bf9b40f0a89c24b553aefbf2', 'request_id': 'req-1373cc75-d0aa-47d9-94ea-860f5fc2008c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.995542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1373cc75-d0aa-47d9-94ea-860f5fc2008c admin admin] The token cache is being invalidate because project feb78b24bf9b40f0a89c24b553aefbf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:12.995879 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1373cc75-d0aa-47d9-94ea-860f5fc2008c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:12.999359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1373cc75-d0aa-47d9-94ea-860f5fc2008c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:13.001223 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1006/1987] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:12 2022] DELETE /identity/v3/projects/feb78b24bf9b40f0a89c24b553aefbf2 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:13.006473 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a17aef-408a-4cf1-81af-e2311474149e None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.006817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a17aef-408a-4cf1-81af-e2311474149e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.007104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a17aef-408a-4cf1-81af-e2311474149e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.025948 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-39a17aef-408a-4cf1-81af-e2311474149e None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:13.027958 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-39a17aef-408a-4cf1-81af-e2311474149e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:13.047749 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:13.115183 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 982/1988] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:13.124996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7c45a-641e-45b9-9cd0-0463807e92c6 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.125514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7c45a-641e-45b9-9cd0-0463807e92c6 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.126011 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf7c45a-641e-45b9-9cd0-0463807e92c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.147891 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-bcf7c45a-641e-45b9-9cd0-0463807e92c6 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:13.149026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-bcf7c45a-641e-45b9-9cd0-0463807e92c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:13.157658 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1007/1989] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:13.162260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a17aef-408a-4cf1-81af-e2311474149e None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:13.172708 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a17aef-408a-4cf1-81af-e2311474149e None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:13.179452 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641342e-4053-4b59-a845-78fb0edf69c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:13.180226 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641342e-4053-4b59-a845-78fb0edf69c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.180555 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641342e-4053-4b59-a845-78fb0edf69c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.180832 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b641342e-4053-4b59-a845-78fb0edf69c2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.181624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641342e-4053-4b59-a845-78fb0edf69c2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:02:13.181937 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641342e-4053-4b59-a845-78fb0edf69c2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:13.183685 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b641342e-4053-4b59-a845-78fb0edf69c2 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:13.198632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 983/1990] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:02:13.199031 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7c45a-641e-45b9-9cd0-0463807e92c6 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:13.200649 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-bcf7c45a-641e-45b9-9cd0-0463807e92c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:13.204792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641342e-4053-4b59-a845-78fb0edf69c2 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:13.217428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b641342e-4053-4b59-a845-78fb0edf69c2 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:13.224060 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451fa82-f607-4c77-b912-6b74ca39b59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:13.224529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf7c45a-641e-45b9-9cd0-0463807e92c6 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:13.225388 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451fa82-f607-4c77-b912-6b74ca39b59d admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.225388 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451fa82-f607-4c77-b912-6b74ca39b59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.225803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b451fa82-f607-4c77-b912-6b74ca39b59d admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.226858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b451fa82-f607-4c77-b912-6b74ca39b59d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:13.229711 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b451fa82-f607-4c77-b912-6b74ca39b59d admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:13.237391 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73052af5-c6f9-4ff1-93de-811a60d724c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:13.238363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73052af5-c6f9-4ff1-93de-811a60d724c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.238769 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73052af5-c6f9-4ff1-93de-811a60d724c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.239141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73052af5-c6f9-4ff1-93de-811a60d724c1 admin admin] PATH_INFO: `/v3/projects/390df83726124500bde5f207ee4b6629` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.240488 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73052af5-c6f9-4ff1-93de-811a60d724c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=390df83726124500bde5f207ee4b6629)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:13.242579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73052af5-c6f9-4ff1-93de-811a60d724c1 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:13.260834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 984/1991] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:13 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:13.264784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451fa82-f607-4c77-b912-6b74ca39b59d admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:13.294170 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-73052af5-c6f9-4ff1-93de-811a60d724c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '390df83726124500bde5f207ee4b6629', 'request_id': 'req-73052af5-c6f9-4ff1-93de-811a60d724c1'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:13.301066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-73052af5-c6f9-4ff1-93de-811a60d724c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '390df83726124500bde5f207ee4b6629', 'request_id': 'req-73052af5-c6f9-4ff1-93de-811a60d724c1'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:13.307115 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-73052af5-c6f9-4ff1-93de-811a60d724c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '390df83726124500bde5f207ee4b6629', 'request_id': 'req-73052af5-c6f9-4ff1-93de-811a60d724c1'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:13.320803 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-73052af5-c6f9-4ff1-93de-811a60d724c1 admin admin] The token cache is being invalidate because project 390df83726124500bde5f207ee4b6629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:13.321107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-73052af5-c6f9-4ff1-93de-811a60d724c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:13.321758 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-73052af5-c6f9-4ff1-93de-811a60d724c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:13.323402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1008/1992] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:13 2022] DELETE /identity/v3/projects/390df83726124500bde5f207ee4b6629 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:02:13.328119 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8ffc3-f1a7-4349-8fd0-d1a72cc17675 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.328561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8ffc3-f1a7-4349-8fd0-d1a72cc17675 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.328846 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba8ffc3-f1a7-4349-8fd0-d1a72cc17675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.336879 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b451fa82-f607-4c77-b912-6b74ca39b59d admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:13.349804 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-eba8ffc3-f1a7-4349-8fd0-d1a72cc17675 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:13.351322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-eba8ffc3-f1a7-4349-8fd0-d1a72cc17675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:13.366721 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:13.414685 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6aafdf-73cf-47c0-aa3c-53898a738f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:13.416550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6aafdf-73cf-47c0-aa3c-53898a738f6d admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.416550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6aafdf-73cf-47c0-aa3c-53898a738f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.416550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6aafdf-73cf-47c0-aa3c-53898a738f6d admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.416785 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6aafdf-73cf-47c0-aa3c-53898a738f6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:13.419069 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6aafdf-73cf-47c0-aa3c-53898a738f6d admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:13.447696 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 985/1993] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:13 2022] POST /identity/v3/users => generated 289 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:02:13.452086 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6aafdf-73cf-47c0-aa3c-53898a738f6d admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:13.454755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-ba6aafdf-73cf-47c0-aa3c-53898a738f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:13.466387 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1009/1994] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:13.474647 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6aafdf-73cf-47c0-aa3c-53898a738f6d admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:13.475218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58995078-e8cc-431f-b777-66a6c33a8865 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.475624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58995078-e8cc-431f-b777-66a6c33a8865 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.476106 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58995078-e8cc-431f-b777-66a6c33a8865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.480354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ad488-7057-4bb7-904c-ab5b17382c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:13.481233 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ad488-7057-4bb7-904c-ab5b17382c4d admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.481727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ad488-7057-4bb7-904c-ab5b17382c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.482025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ad488-7057-4bb7-904c-ab5b17382c4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.482792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ad488-7057-4bb7-904c-ab5b17382c4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:13.485789 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279ad488-7057-4bb7-904c-ab5b17382c4d admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:13.496368 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-58995078-e8cc-431f-b777-66a6c33a8865 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:13.497716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-58995078-e8cc-431f-b777-66a6c33a8865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:13.500608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 986/1995] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:13.505630 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ad488-7057-4bb7-904c-ab5b17382c4d admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:13.514161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1010/1996] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:13.518194 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58995078-e8cc-431f-b777-66a6c33a8865 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:13.520672 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279ad488-7057-4bb7-904c-ab5b17382c4d admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:13.525913 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb20b3a-075b-4c9e-b3a5-5c34452d7ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:13.527503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb20b3a-075b-4c9e-b3a5-5c34452d7ec7 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.527793 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb20b3a-075b-4c9e-b3a5-5c34452d7ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.528074 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb20b3a-075b-4c9e-b3a5-5c34452d7ec7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.528797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb20b3a-075b-4c9e-b3a5-5c34452d7ec7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:13.530438 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb20b3a-075b-4c9e-b3a5-5c34452d7ec7 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:13.533257 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:13.536049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58995078-e8cc-431f-b777-66a6c33a8865 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:13.541136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b3d50-aadf-40ef-9089-cd54ec0dc3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:13.541888 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b3d50-aadf-40ef-9089-cd54ec0dc3fe admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.542259 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b3d50-aadf-40ef-9089-cd54ec0dc3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.542693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398b3d50-aadf-40ef-9089-cd54ec0dc3fe admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.543469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b3d50-aadf-40ef-9089-cd54ec0dc3fe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:02:13.543849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b3d50-aadf-40ef-9089-cd54ec0dc3fe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:13.545818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 987/1997] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:13.547044 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398b3d50-aadf-40ef-9089-cd54ec0dc3fe admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:13.549391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb20b3a-075b-4c9e-b3a5-5c34452d7ec7 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:13.559930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb20b3a-075b-4c9e-b3a5-5c34452d7ec7 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:13.562215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1011/1998] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:02:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 23:02:13.565418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134db9bf-b627-4a25-b074-25d2a82f23b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:13.566456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134db9bf-b627-4a25-b074-25d2a82f23b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.566881 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134db9bf-b627-4a25-b074-25d2a82f23b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.567161 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134db9bf-b627-4a25-b074-25d2a82f23b4 admin admin] PATH_INFO: `/v3/projects/192ad3f1e3c84e1b923d74cac0f62fbf/users/910ef56632b3482699a33a6157c8d289/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.567289 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b3d50-aadf-40ef-9089-cd54ec0dc3fe admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:13.580327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398b3d50-aadf-40ef-9089-cd54ec0dc3fe admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:13.583256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134db9bf-b627-4a25-b074-25d2a82f23b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=192ad3f1e3c84e1b923d74cac0f62fbf, user_id=910ef56632b3482699a33a6157c8d289, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:13.584717 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8c701-abbb-49d8-ae85-8797ef7cd543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:13.585464 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8c701-abbb-49d8-ae85-8797ef7cd543 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.585852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8c701-abbb-49d8-ae85-8797ef7cd543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.585960 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134db9bf-b627-4a25-b074-25d2a82f23b4 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:13.586217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c8c701-abbb-49d8-ae85-8797ef7cd543 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.587485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8c701-abbb-49d8-ae85-8797ef7cd543 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:13.589986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c8c701-abbb-49d8-ae85-8797ef7cd543 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:13.604414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 988/1999] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:13 2022] PUT /identity/v3/projects/192ad3f1e3c84e1b923d74cac0f62fbf/users/910ef56632b3482699a33a6157c8d289/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:02:13.614725 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d296ad-d20c-43e2-a1df-3dc6492ae60f None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.615131 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d296ad-d20c-43e2-a1df-3dc6492ae60f None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.615508 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d296ad-d20c-43e2-a1df-3dc6492ae60f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.634204 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1012/2000] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:13 2022] POST /identity/v3/projects => generated 403 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:13.638097 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8c701-abbb-49d8-ae85-8797ef7cd543 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:13.644471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-e3d296ad-d20c-43e2-a1df-3dc6492ae60f None None] MFA Rules not processed for user `910ef56632b3482699a33a6157c8d289`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:13.646118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-e3d296ad-d20c-43e2-a1df-3dc6492ae60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:13.709108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c8c701-abbb-49d8-ae85-8797ef7cd543 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:13.716535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044d3eb-fe7c-4f6a-8b68-364f3ec15ac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:13.717244 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044d3eb-fe7c-4f6a-8b68-364f3ec15ac4 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.717537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044d3eb-fe7c-4f6a-8b68-364f3ec15ac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.717824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c044d3eb-fe7c-4f6a-8b68-364f3ec15ac4 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.718740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044d3eb-fe7c-4f6a-8b68-364f3ec15ac4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:13.720822 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c044d3eb-fe7c-4f6a-8b68-364f3ec15ac4 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:13.730994 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 989/2001] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:13 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:13.744143 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d296ad-d20c-43e2-a1df-3dc6492ae60f None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:13.746233 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-e3d296ad-d20c-43e2-a1df-3dc6492ae60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:13.753783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1013/2002] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:13 2022] POST /identity/v3/users => generated 302 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:02:13.754855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044d3eb-fe7c-4f6a-8b68-364f3ec15ac4 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:13.765927 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c044d3eb-fe7c-4f6a-8b68-364f3ec15ac4 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:13.770743 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd70d62-c59f-4cc5-89b4-9c26809c5640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:13.771404 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd70d62-c59f-4cc5-89b4-9c26809c5640 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.771698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd70d62-c59f-4cc5-89b4-9c26809c5640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.771984 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd70d62-c59f-4cc5-89b4-9c26809c5640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.772711 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd70d62-c59f-4cc5-89b4-9c26809c5640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:13.774341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd70d62-c59f-4cc5-89b4-9c26809c5640 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:13.785020 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1014/2003] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:02:13.788913 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd70d62-c59f-4cc5-89b4-9c26809c5640 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:13.799749 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd70d62-c59f-4cc5-89b4-9c26809c5640 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:13.805066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f624445-568c-4227-9e51-a1218c2e5f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:13.805719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f624445-568c-4227-9e51-a1218c2e5f68 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.806011 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f624445-568c-4227-9e51-a1218c2e5f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.806328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f624445-568c-4227-9e51-a1218c2e5f68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.807096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f624445-568c-4227-9e51-a1218c2e5f68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:13.807494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d296ad-d20c-43e2-a1df-3dc6492ae60f None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:13.808741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f624445-568c-4227-9e51-a1218c2e5f68 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:13.811406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc5ebb0-e9ba-4029-b350-f9aedeb55336 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915f4296504b47d796e3a5b99550eb75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:13.812151 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5ebb0-e9ba-4029-b350-f9aedeb55336 service glance] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.812525 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5ebb0-e9ba-4029-b350-f9aedeb55336 service glance] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.813526 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5ebb0-e9ba-4029-b350-f9aedeb55336 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.820125 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1015/2004] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:02:13.822542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5ebb0-e9ba-4029-b350-f9aedeb55336 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:13.824011 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f624445-568c-4227-9e51-a1218c2e5f68 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:13.824141 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc5ebb0-e9ba-4029-b350-f9aedeb55336 service glance] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:13.833597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 990/2005] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:13.834718 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f624445-568c-4227-9e51-a1218c2e5f68 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:13.841087 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533de526-525f-41ee-a060-178778253071 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:13.841990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533de526-525f-41ee-a060-178778253071 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.842288 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533de526-525f-41ee-a060-178778253071 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.842570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533de526-525f-41ee-a060-178778253071 admin admin] PATH_INFO: `/v3/projects/d0ae14018002474c94f643ab57fb936d/users/29aba8869dc84c3c94bc15294c8a9ed2/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.863223 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533de526-525f-41ee-a060-178778253071 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ae14018002474c94f643ab57fb936d, user_id=29aba8869dc84c3c94bc15294c8a9ed2, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:13.865878 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533de526-525f-41ee-a060-178778253071 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:13.880557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1016/2006] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:13 2022] PUT /identity/v3/projects/d0ae14018002474c94f643ab57fb936d/users/29aba8869dc84c3c94bc15294c8a9ed2/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:02:13.890792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3cdee-9fb9-4beb-aa83-a8d1b069c7f1 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:13.891095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3cdee-9fb9-4beb-aa83-a8d1b069c7f1 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:13.891705 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd3cdee-9fb9-4beb-aa83-a8d1b069c7f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:13.929152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-9fd3cdee-9fb9-4beb-aa83-a8d1b069c7f1 None None] MFA Rules not processed for user `29aba8869dc84c3c94bc15294c8a9ed2`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:13.931213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-9fd3cdee-9fb9-4beb-aa83-a8d1b069c7f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:14.001020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:14.014078 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 991/2007] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:14.018220 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533de526-525f-41ee-a060-178778253071 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:14.033627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:14.114434 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533de526-525f-41ee-a060-178778253071 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:14.120437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453307d-1fbe-4253-a028-92d3b11598c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:14.121583 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453307d-1fbe-4253-a028-92d3b11598c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:14.122101 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453307d-1fbe-4253-a028-92d3b11598c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:14.122460 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5453307d-1fbe-4253-a028-92d3b11598c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:14.123275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453307d-1fbe-4253-a028-92d3b11598c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:14.125814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5453307d-1fbe-4253-a028-92d3b11598c8 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:14.183670 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1017/2008] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:14 2022] POST /identity/v3/projects => generated 405 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:14.187687 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3cdee-9fb9-4beb-aa83-a8d1b069c7f1 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:14.213412 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd3cdee-9fb9-4beb-aa83-a8d1b069c7f1 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:14.314082 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d2be8-56e6-4f47-889b-e6e85b3724d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:14.315155 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d2be8-56e6-4f47-889b-e6e85b3724d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:14.315800 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d2be8-56e6-4f47-889b-e6e85b3724d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:14.315914 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780d2be8-56e6-4f47-889b-e6e85b3724d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:14.316668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d2be8-56e6-4f47-889b-e6e85b3724d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:14.319315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780d2be8-56e6-4f47-889b-e6e85b3724d8 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:14.345363 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 992/2009] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:14 2022] POST /identity/v3/users => generated 309 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:02:14.346940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453307d-1fbe-4253-a028-92d3b11598c8 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:14.368515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5453307d-1fbe-4253-a028-92d3b11598c8 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:14.374454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b516eb9-fc01-4e13-b544-64b10021deb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:14.375181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b516eb9-fc01-4e13-b544-64b10021deb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:14.375478 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b516eb9-fc01-4e13-b544-64b10021deb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:14.375766 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b516eb9-fc01-4e13-b544-64b10021deb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:14.376506 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b516eb9-fc01-4e13-b544-64b10021deb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:14.378120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b516eb9-fc01-4e13-b544-64b10021deb8 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:14.395157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1018/2010] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:02:14.400530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d2be8-56e6-4f47-889b-e6e85b3724d8 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:14.412326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780d2be8-56e6-4f47-889b-e6e85b3724d8 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:14.417072 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1afcf3-be93-4623-a699-3e295b71681d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:14.418826 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1afcf3-be93-4623-a699-3e295b71681d admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:14.419449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1afcf3-be93-4623-a699-3e295b71681d admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:14.420038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1afcf3-be93-4623-a699-3e295b71681d admin admin] PATH_INFO: `/v3/projects/8691e38b19934c4181683d4c88ae2e7b/users/dd27642ca5c34b8189e77a5d8097ae56/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:14.437160 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1afcf3-be93-4623-a699-3e295b71681d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8691e38b19934c4181683d4c88ae2e7b, user_id=dd27642ca5c34b8189e77a5d8097ae56, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:14.440111 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1afcf3-be93-4623-a699-3e295b71681d admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:14.454488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 993/2011] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:14 2022] PUT /identity/v3/projects/8691e38b19934c4181683d4c88ae2e7b/users/dd27642ca5c34b8189e77a5d8097ae56/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:14.458192 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b516eb9-fc01-4e13-b544-64b10021deb8 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:14.469498 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b516eb9-fc01-4e13-b544-64b10021deb8 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:14.550123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e9b5da-4e47-43a8-b03e-27a3db230f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:14.550958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9b5da-4e47-43a8-b03e-27a3db230f3a admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:14.551324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9b5da-4e47-43a8-b03e-27a3db230f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:14.551613 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e9b5da-4e47-43a8-b03e-27a3db230f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:14.552841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9b5da-4e47-43a8-b03e-27a3db230f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:14.554476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e9b5da-4e47-43a8-b03e-27a3db230f3a admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:14.573476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1019/2012] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:14 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:14.575006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1afcf3-be93-4623-a699-3e295b71681d admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:14.598943 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1afcf3-be93-4623-a699-3e295b71681d admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:14.609791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5451e-da31-4025-b1da-67089efad7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:14.612124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5451e-da31-4025-b1da-67089efad7fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:14.612424 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5451e-da31-4025-b1da-67089efad7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:14.615104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5451e-da31-4025-b1da-67089efad7fb admin admin] PATH_INFO: `/v3/projects/8691e38b19934c4181683d4c88ae2e7b/users/dd27642ca5c34b8189e77a5d8097ae56/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:14.622855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5451e-da31-4025-b1da-67089efad7fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8691e38b19934c4181683d4c88ae2e7b, user_id=dd27642ca5c34b8189e77a5d8097ae56, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:14.627357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5451e-da31-4025-b1da-67089efad7fb admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:14.647440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 994/2013] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:14 2022] PUT /identity/v3/projects/8691e38b19934c4181683d4c88ae2e7b/users/dd27642ca5c34b8189e77a5d8097ae56/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:14.660549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2d949-2457-4a5c-bee8-e7986674896e None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:14.660549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2d949-2457-4a5c-bee8-e7986674896e None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:14.660772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d2d949-2457-4a5c-bee8-e7986674896e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:14.705577 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-06d2d949-2457-4a5c-bee8-e7986674896e None None] MFA Rules not processed for user `dd27642ca5c34b8189e77a5d8097ae56`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:14.706768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-06d2d949-2457-4a5c-bee8-e7986674896e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:14.790116 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1020/2014] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:14 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:14.801783 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5451e-da31-4025-b1da-67089efad7fb admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:14.803656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-39e5451e-da31-4025-b1da-67089efad7fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:14.892869 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5451e-da31-4025-b1da-67089efad7fb admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:14.897315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e810e4cf-939a-4475-a20e-b053164052fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:14.898512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810e4cf-939a-4475-a20e-b053164052fc service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:14.898512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810e4cf-939a-4475-a20e-b053164052fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:14.898792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e810e4cf-939a-4475-a20e-b053164052fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:14.911819 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e810e4cf-939a-4475-a20e-b053164052fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:14.913382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e810e4cf-939a-4475-a20e-b053164052fc service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:14.925952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 995/2015] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:14 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:02:15.067767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2d949-2457-4a5c-bee8-e7986674896e None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:15.080200 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:15.158157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d2d949-2457-4a5c-bee8-e7986674896e None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:15.168412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c524079-bef6-4da8-a152-88edb67a77b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:15.168791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c524079-bef6-4da8-a152-88edb67a77b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:15.169109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c524079-bef6-4da8-a152-88edb67a77b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:15.169433 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c524079-bef6-4da8-a152-88edb67a77b5 admin admin] PATH_INFO: `/v3/users/29aba8869dc84c3c94bc15294c8a9ed2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:15.170755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c524079-bef6-4da8-a152-88edb67a77b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29aba8869dc84c3c94bc15294c8a9ed2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:15.180972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c524079-bef6-4da8-a152-88edb67a77b5 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:15.256079 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-0c524079-bef6-4da8-a152-88edb67a77b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29aba8869dc84c3c94bc15294c8a9ed2'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:15.265542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-0c524079-bef6-4da8-a152-88edb67a77b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29aba8869dc84c3c94bc15294c8a9ed2'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:15.266290 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-0c524079-bef6-4da8-a152-88edb67a77b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29aba8869dc84c3c94bc15294c8a9ed2'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:15.285073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-0c524079-bef6-4da8-a152-88edb67a77b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29aba8869dc84c3c94bc15294c8a9ed2'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:15.285800 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-0c524079-bef6-4da8-a152-88edb67a77b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29aba8869dc84c3c94bc15294c8a9ed2'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:15.305679 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1021/2016] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:15 2022] DELETE /identity/v3/users/29aba8869dc84c3c94bc15294c8a9ed2 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:15.310945 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86436ab9-db01-432c-977d-9a374e9ae39e None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:15.311378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86436ab9-db01-432c-977d-9a374e9ae39e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:15.311753 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86436ab9-db01-432c-977d-9a374e9ae39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:15.332570 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-86436ab9-db01-432c-977d-9a374e9ae39e None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:15.334063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-86436ab9-db01-432c-977d-9a374e9ae39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:15.407082 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:15.428371 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 996/2017] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:15.458688 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c524079-bef6-4da8-a152-88edb67a77b5 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:15.461318 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-0c524079-bef6-4da8-a152-88edb67a77b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:15.579941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c524079-bef6-4da8-a152-88edb67a77b5 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:15.583982 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe7376-905f-41c1-b738-94312c7832d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:15.584557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe7376-905f-41c1-b738-94312c7832d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:15.584850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe7376-905f-41c1-b738-94312c7832d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:15.585136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe7376-905f-41c1-b738-94312c7832d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:15.595787 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe7376-905f-41c1-b738-94312c7832d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:15.597238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffe7376-905f-41c1-b738-94312c7832d4 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:15.608953 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1022/2018] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:16.076322 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86436ab9-db01-432c-977d-9a374e9ae39e None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:16.077785 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-86436ab9-db01-432c-977d-9a374e9ae39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:16.098424 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86436ab9-db01-432c-977d-9a374e9ae39e None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:16.108628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43951cd8-4cfc-472c-8dc1-d858a8267da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:16.109758 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43951cd8-4cfc-472c-8dc1-d858a8267da9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:16.110223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43951cd8-4cfc-472c-8dc1-d858a8267da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:16.110545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43951cd8-4cfc-472c-8dc1-d858a8267da9 admin admin] PATH_INFO: `/v3/projects/d0ae14018002474c94f643ab57fb936d` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:16.112104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43951cd8-4cfc-472c-8dc1-d858a8267da9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0ae14018002474c94f643ab57fb936d)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:16.114396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43951cd8-4cfc-472c-8dc1-d858a8267da9 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:16.161866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-43951cd8-4cfc-472c-8dc1-d858a8267da9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0ae14018002474c94f643ab57fb936d', 'request_id': 'req-43951cd8-4cfc-472c-8dc1-d858a8267da9'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:16.178045 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-43951cd8-4cfc-472c-8dc1-d858a8267da9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0ae14018002474c94f643ab57fb936d', 'request_id': 'req-43951cd8-4cfc-472c-8dc1-d858a8267da9'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:16.184491 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-43951cd8-4cfc-472c-8dc1-d858a8267da9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0ae14018002474c94f643ab57fb936d', 'request_id': 'req-43951cd8-4cfc-472c-8dc1-d858a8267da9'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:16.198561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-43951cd8-4cfc-472c-8dc1-d858a8267da9 admin admin] The token cache is being invalidate because project d0ae14018002474c94f643ab57fb936d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:16.198878 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-43951cd8-4cfc-472c-8dc1-d858a8267da9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:16.199495 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-43951cd8-4cfc-472c-8dc1-d858a8267da9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:16.201025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 997/2019] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:16 2022] DELETE /identity/v3/projects/d0ae14018002474c94f643ab57fb936d => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:16.205071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe7376-905f-41c1-b738-94312c7832d4 service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:16.215323 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe7376-905f-41c1-b738-94312c7832d4 service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:16.243178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:16.275330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac77d59-e87b-4fda-a3e6-7e1ec4f83e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:16.276152 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac77d59-e87b-4fda-a3e6-7e1ec4f83e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:16.276437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac77d59-e87b-4fda-a3e6-7e1ec4f83e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:16.277225 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac77d59-e87b-4fda-a3e6-7e1ec4f83e5e admin admin] PATH_INFO: `/v3/users/dd27642ca5c34b8189e77a5d8097ae56` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:16.278393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac77d59-e87b-4fda-a3e6-7e1ec4f83e5e admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd27642ca5c34b8189e77a5d8097ae56)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:16.280429 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac77d59-e87b-4fda-a3e6-7e1ec4f83e5e admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:16.329196 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bac77d59-e87b-4fda-a3e6-7e1ec4f83e5e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd27642ca5c34b8189e77a5d8097ae56'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:16.336134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bac77d59-e87b-4fda-a3e6-7e1ec4f83e5e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd27642ca5c34b8189e77a5d8097ae56'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:16.336819 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bac77d59-e87b-4fda-a3e6-7e1ec4f83e5e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd27642ca5c34b8189e77a5d8097ae56'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:16.349605 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bac77d59-e87b-4fda-a3e6-7e1ec4f83e5e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd27642ca5c34b8189e77a5d8097ae56'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:16.350234 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bac77d59-e87b-4fda-a3e6-7e1ec4f83e5e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd27642ca5c34b8189e77a5d8097ae56'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:16.363386 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1023/2020] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:16 2022] DELETE /identity/v3/users/dd27642ca5c34b8189e77a5d8097ae56 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:16.642495 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43951cd8-4cfc-472c-8dc1-d858a8267da9 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:16.644260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-43951cd8-4cfc-472c-8dc1-d858a8267da9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:16.658023 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43951cd8-4cfc-472c-8dc1-d858a8267da9 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:16.724048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f200aad-aa0b-49ff-8b6e-24bcde5d0a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:16.724926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f200aad-aa0b-49ff-8b6e-24bcde5d0a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:16.725213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f200aad-aa0b-49ff-8b6e-24bcde5d0a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:16.725496 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f200aad-aa0b-49ff-8b6e-24bcde5d0a4e admin admin] PATH_INFO: `/v3/projects/8691e38b19934c4181683d4c88ae2e7b` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:16.727020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f200aad-aa0b-49ff-8b6e-24bcde5d0a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=8691e38b19934c4181683d4c88ae2e7b)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:16.728981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f200aad-aa0b-49ff-8b6e-24bcde5d0a4e admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:16.776176 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6f200aad-aa0b-49ff-8b6e-24bcde5d0a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8691e38b19934c4181683d4c88ae2e7b', 'request_id': 'req-6f200aad-aa0b-49ff-8b6e-24bcde5d0a4e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:16.782691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6f200aad-aa0b-49ff-8b6e-24bcde5d0a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8691e38b19934c4181683d4c88ae2e7b', 'request_id': 'req-6f200aad-aa0b-49ff-8b6e-24bcde5d0a4e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:16.788154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6f200aad-aa0b-49ff-8b6e-24bcde5d0a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8691e38b19934c4181683d4c88ae2e7b', 'request_id': 'req-6f200aad-aa0b-49ff-8b6e-24bcde5d0a4e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:16.801634 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6f200aad-aa0b-49ff-8b6e-24bcde5d0a4e admin admin] The token cache is being invalidate because project 8691e38b19934c4181683d4c88ae2e7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:16.801928 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6f200aad-aa0b-49ff-8b6e-24bcde5d0a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:16.802575 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6f200aad-aa0b-49ff-8b6e-24bcde5d0a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:16.804331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 998/2021] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:16 2022] DELETE /identity/v3/projects/8691e38b19934c4181683d4c88ae2e7b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:16.809647 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0575a9-3061-4a27-98b6-e9e19b3387eb None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:16.809949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0575a9-3061-4a27-98b6-e9e19b3387eb None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:16.810233 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0575a9-3061-4a27-98b6-e9e19b3387eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:16.828498 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-ea0575a9-3061-4a27-98b6-e9e19b3387eb None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:16.829717 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ea0575a9-3061-4a27-98b6-e9e19b3387eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:16.889869 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1024/2022] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:16.897498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f8dc7-7b49-484a-9c3c-523887d5828d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:16.897790 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f8dc7-7b49-484a-9c3c-523887d5828d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:16.898072 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f8dc7-7b49-484a-9c3c-523887d5828d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:16.905743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:16.919162 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-f45f8dc7-7b49-484a-9c3c-523887d5828d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:16.920380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f45f8dc7-7b49-484a-9c3c-523887d5828d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:16.928328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 999/2023] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:16.933596 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0575a9-3061-4a27-98b6-e9e19b3387eb None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:16.943562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0575a9-3061-4a27-98b6-e9e19b3387eb None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:16.948119 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3270ac-b48a-4810-bf2c-56e948727c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:16.948807 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3270ac-b48a-4810-bf2c-56e948727c25 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:16.949092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3270ac-b48a-4810-bf2c-56e948727c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:16.949372 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3270ac-b48a-4810-bf2c-56e948727c25 admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:16.950105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3270ac-b48a-4810-bf2c-56e948727c25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:02:16.950517 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3270ac-b48a-4810-bf2c-56e948727c25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:16.952224 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3270ac-b48a-4810-bf2c-56e948727c25 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:16.967936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1025/2024] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:02:16.968984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f8dc7-7b49-484a-9c3c-523887d5828d None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:16.978804 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f8dc7-7b49-484a-9c3c-523887d5828d None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:16.984330 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2530a-d25e-4cb7-815e-20b45ad007e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:16.985038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2530a-d25e-4cb7-815e-20b45ad007e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:16.985449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2530a-d25e-4cb7-815e-20b45ad007e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:16.985734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e2530a-d25e-4cb7-815e-20b45ad007e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:16.986440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2530a-d25e-4cb7-815e-20b45ad007e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:16.988391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e2530a-d25e-4cb7-815e-20b45ad007e4 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:17.015537 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1000/2025] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:16 2022] POST /identity/v3/projects => generated 357 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:02:17.019835 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3270ac-b48a-4810-bf2c-56e948727c25 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:17.030831 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3270ac-b48a-4810-bf2c-56e948727c25 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:17.067887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:17.094322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229acf4-0e1d-4705-80b3-b504cb9b59a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:17.095038 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229acf4-0e1d-4705-80b3-b504cb9b59a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:17.095331 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229acf4-0e1d-4705-80b3-b504cb9b59a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:17.095618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d229acf4-0e1d-4705-80b3-b504cb9b59a1 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:17.096309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229acf4-0e1d-4705-80b3-b504cb9b59a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:17.098224 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d229acf4-0e1d-4705-80b3-b504cb9b59a1 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:17.118911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1026/2026] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:17 2022] POST /identity/v3/users => generated 279 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:17.123552 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2530a-d25e-4cb7-815e-20b45ad007e4 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:17.135774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e2530a-d25e-4cb7-815e-20b45ad007e4 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:17.151289 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a0747-54ff-4759-8369-bdee542f0ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:17.152761 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a0747-54ff-4759-8369-bdee542f0ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:17.153368 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a0747-54ff-4759-8369-bdee542f0ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:17.153814 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a0747-54ff-4759-8369-bdee542f0ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:17.154619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a0747-54ff-4759-8369-bdee542f0ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:17.156334 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a0747-54ff-4759-8369-bdee542f0ea3 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:17.168536 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1001/2027] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:17.172393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229acf4-0e1d-4705-80b3-b504cb9b59a1 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:17.183646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d229acf4-0e1d-4705-80b3-b504cb9b59a1 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:17.188572 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89e81fc-0ca5-4103-9002-52bf1c11cfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:17.189255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e81fc-0ca5-4103-9002-52bf1c11cfc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:17.189548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e81fc-0ca5-4103-9002-52bf1c11cfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:17.189831 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89e81fc-0ca5-4103-9002-52bf1c11cfc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:17.191048 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89e81fc-0ca5-4103-9002-52bf1c11cfc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:17.192572 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89e81fc-0ca5-4103-9002-52bf1c11cfc3 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:17.207262 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1027/2028] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:17.211825 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a0747-54ff-4759-8369-bdee542f0ea3 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:17.224238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a0747-54ff-4759-8369-bdee542f0ea3 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:17.236223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7b5b4-9165-4c56-a2ac-a5c7f576468b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:17.237668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7b5b4-9165-4c56-a2ac-a5c7f576468b admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:17.237960 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7b5b4-9165-4c56-a2ac-a5c7f576468b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:17.238341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc7b5b4-9165-4c56-a2ac-a5c7f576468b admin admin] PATH_INFO: `/v3/projects/850f9eb2a4b442d8ae329065e0e30a27/users/95029fe88d4141138ea871ef340631a4/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:17.254577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7b5b4-9165-4c56-a2ac-a5c7f576468b admin admin] RBAC: Authorizing `identity:create_grant(project_id=850f9eb2a4b442d8ae329065e0e30a27, user_id=95029fe88d4141138ea871ef340631a4, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:17.257673 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc7b5b4-9165-4c56-a2ac-a5c7f576468b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:17.274140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1002/2029] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:17 2022] PUT /identity/v3/projects/850f9eb2a4b442d8ae329065e0e30a27/users/95029fe88d4141138ea871ef340631a4/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:17.285553 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d02d6b-4bce-4687-9da9-b59835db7100 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:17.285810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d02d6b-4bce-4687-9da9-b59835db7100 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:17.286090 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d02d6b-4bce-4687-9da9-b59835db7100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:17.317488 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-16d02d6b-4bce-4687-9da9-b59835db7100 None None] MFA Rules not processed for user `95029fe88d4141138ea871ef340631a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:17.318745 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-16d02d6b-4bce-4687-9da9-b59835db7100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:17.391016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1028/2030] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:17 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:17.393048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7b5b4-9165-4c56-a2ac-a5c7f576468b admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:17.460669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc7b5b4-9165-4c56-a2ac-a5c7f576468b admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:17.466007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0239e-696d-4b64-9559-50430a8031fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:17.466708 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0239e-696d-4b64-9559-50430a8031fe admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:17.467001 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0239e-696d-4b64-9559-50430a8031fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:17.467281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb0239e-696d-4b64-9559-50430a8031fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:17.468003 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb0239e-696d-4b64-9559-50430a8031fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:17.469963 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb0239e-696d-4b64-9559-50430a8031fe admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:17.494946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1003/2031] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:17 2022] POST /identity/v3/projects => generated 357 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:17.499211 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d02d6b-4bce-4687-9da9-b59835db7100 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:17.511739 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d02d6b-4bce-4687-9da9-b59835db7100 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:17.574445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a74ff3-fc0d-46c8-a82b-9d5b8bd2770a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:17.575076 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a74ff3-fc0d-46c8-a82b-9d5b8bd2770a admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:17.575688 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a74ff3-fc0d-46c8-a82b-9d5b8bd2770a admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:17.575979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a74ff3-fc0d-46c8-a82b-9d5b8bd2770a admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:17.576790 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a74ff3-fc0d-46c8-a82b-9d5b8bd2770a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:17.578739 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a74ff3-fc0d-46c8-a82b-9d5b8bd2770a admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:17.602890 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1029/2032] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:17 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:17.607296 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0239e-696d-4b64-9559-50430a8031fe admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:17.622903 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb0239e-696d-4b64-9559-50430a8031fe admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:17.628800 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0840f6-3981-49d5-8c81-aecc1e87716a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:17.629485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0840f6-3981-49d5-8c81-aecc1e87716a admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:17.629817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0840f6-3981-49d5-8c81-aecc1e87716a admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:17.630304 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0840f6-3981-49d5-8c81-aecc1e87716a admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:17.631657 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0840f6-3981-49d5-8c81-aecc1e87716a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:17.634734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0840f6-3981-49d5-8c81-aecc1e87716a admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:17.647970 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1004/2033] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:17.651883 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a74ff3-fc0d-46c8-a82b-9d5b8bd2770a admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:17.662847 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a74ff3-fc0d-46c8-a82b-9d5b8bd2770a admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:17.668918 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba666c0a-42a2-410d-be91-d99ee8634dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:17.669756 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba666c0a-42a2-410d-be91-d99ee8634dd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:17.670043 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba666c0a-42a2-410d-be91-d99ee8634dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:17.670327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba666c0a-42a2-410d-be91-d99ee8634dd1 admin admin] PATH_INFO: `/v3/projects/699bffbce44b4a1586e722fc867b6e2e/users/109311beeb944f36a72df99eff2de8b1/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:17.686909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba666c0a-42a2-410d-be91-d99ee8634dd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=699bffbce44b4a1586e722fc867b6e2e, user_id=109311beeb944f36a72df99eff2de8b1, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:17.690210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba666c0a-42a2-410d-be91-d99ee8634dd1 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:17.704432 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1030/2034] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:17 2022] PUT /identity/v3/projects/699bffbce44b4a1586e722fc867b6e2e/users/109311beeb944f36a72df99eff2de8b1/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:17.708338 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0840f6-3981-49d5-8c81-aecc1e87716a admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:17.722811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0840f6-3981-49d5-8c81-aecc1e87716a admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:17.786069 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a711b923-7e94-4144-ba05-6de538c2e8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:17.786342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711b923-7e94-4144-ba05-6de538c2e8b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:17.786655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711b923-7e94-4144-ba05-6de538c2e8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:17.786932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a711b923-7e94-4144-ba05-6de538c2e8b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:17.787663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711b923-7e94-4144-ba05-6de538c2e8b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:17.789246 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a711b923-7e94-4144-ba05-6de538c2e8b5 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:17.800017 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1005/2035] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:17.803811 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba666c0a-42a2-410d-be91-d99ee8634dd1 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:17.815193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba666c0a-42a2-410d-be91-d99ee8634dd1 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:17.822854 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50fd97-a3fe-40e3-bd86-404bef52de1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:17.824310 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50fd97-a3fe-40e3-bd86-404bef52de1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:17.824701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50fd97-a3fe-40e3-bd86-404bef52de1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:17.824907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b50fd97-a3fe-40e3-bd86-404bef52de1b admin admin] PATH_INFO: `/v3/projects/699bffbce44b4a1586e722fc867b6e2e/users/109311beeb944f36a72df99eff2de8b1/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:17.826965 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50fd97-a3fe-40e3-bd86-404bef52de1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=699bffbce44b4a1586e722fc867b6e2e, user_id=109311beeb944f36a72df99eff2de8b1, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:17.829630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b50fd97-a3fe-40e3-bd86-404bef52de1b admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:17.832934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:17.841562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1031/2036] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:17 2022] PUT /identity/v3/projects/699bffbce44b4a1586e722fc867b6e2e/users/109311beeb944f36a72df99eff2de8b1/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:17.853383 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9d9b8-1dd9-4094-a7cb-9cc48e0a3947 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:17.853684 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9d9b8-1dd9-4094-a7cb-9cc48e0a3947 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:17.853975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b9d9b8-1dd9-4094-a7cb-9cc48e0a3947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:17.874217 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:17.884241 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-c7b9d9b8-1dd9-4094-a7cb-9cc48e0a3947 None None] MFA Rules not processed for user `109311beeb944f36a72df99eff2de8b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:17.885631 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c7b9d9b8-1dd9-4094-a7cb-9cc48e0a3947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:17.958984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1006/2037] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:17 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:17.968060 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50fd97-a3fe-40e3-bd86-404bef52de1b admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:17.969926 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7b50fd97-a3fe-40e3-bd86-404bef52de1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:18.033158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b50fd97-a3fe-40e3-bd86-404bef52de1b admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:18.036854 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b8853e-c6ca-462d-967d-0ee16ad9dd8f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:18.037497 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8853e-c6ca-462d-967d-0ee16ad9dd8f service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:18.037781 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8853e-c6ca-462d-967d-0ee16ad9dd8f service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:18.038065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b8853e-c6ca-462d-967d-0ee16ad9dd8f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:18.049788 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8853e-c6ca-462d-967d-0ee16ad9dd8f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:18.051580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b8853e-c6ca-462d-967d-0ee16ad9dd8f service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:18.063460 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1032/2038] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:17 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:18.139045 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9d9b8-1dd9-4094-a7cb-9cc48e0a3947 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:18.148849 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b9d9b8-1dd9-4094-a7cb-9cc48e0a3947 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:18.152629 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66b110-7343-4c62-a4cf-0a5d108216d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:18.153360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66b110-7343-4c62-a4cf-0a5d108216d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:18.153767 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66b110-7343-4c62-a4cf-0a5d108216d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:18.154152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c66b110-7343-4c62-a4cf-0a5d108216d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:18.218946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66b110-7343-4c62-a4cf-0a5d108216d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:18.220404 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c66b110-7343-4c62-a4cf-0a5d108216d9 service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:18.230096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1007/2039] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:18 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:28.058736 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=28 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:28.225197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=57 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:30.893687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14963b-b609-4849-b6c1-dd14e34c265e None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:30.894169 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14963b-b609-4849-b6c1-dd14e34c265e None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:30.894258 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a14963b-b609-4849-b6c1-dd14e34c265e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:30.911930 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-4a14963b-b609-4849-b6c1-dd14e34c265e None None] MFA Rules not processed for user `faf778e713b54a638c1db4a5db6180b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:30.913127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-4a14963b-b609-4849-b6c1-dd14e34c265e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:30.967563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1033/2040] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 23:02:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:31.014533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66b110-7343-4c62-a4cf-0a5d108216d9 service cinder] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:31.018450 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-8c66b110-7343-4c62-a4cf-0a5d108216d9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:31.094516 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c66b110-7343-4c62-a4cf-0a5d108216d9 service cinder] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:31.099573 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f47863-ff05-4755-afb5-b6aa45736a9c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef518c9430e4274b1066241f821f4a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:31.100240 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f47863-ff05-4755-afb5-b6aa45736a9c service swift] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:31.100785 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f47863-ff05-4755-afb5-b6aa45736a9c service swift] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:31.101072 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f47863-ff05-4755-afb5-b6aa45736a9c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:31.110093 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f47863-ff05-4755-afb5-b6aa45736a9c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:31.111714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f47863-ff05-4755-afb5-b6aa45736a9c service swift] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:31.120636 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1008/2041] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 23:02:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 23:02:31.903119 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed5637-06d1-4b4f-bf29-9fbea06a29b3 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:31.904052 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed5637-06d1-4b4f-bf29-9fbea06a29b3 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:31.904552 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ed5637-06d1-4b4f-bf29-9fbea06a29b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:31.925314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-f9ed5637-06d1-4b4f-bf29-9fbea06a29b3 None None] MFA Rules not processed for user `faf778e713b54a638c1db4a5db6180b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:31.926682 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-f9ed5637-06d1-4b4f-bf29-9fbea06a29b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:31.934962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1034/2042] 10.222.0.17 () {58 vars in 967 bytes} [Thu Jun 16 23:02:31 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:31.950770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f47863-ff05-4755-afb5-b6aa45736a9c service swift] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:31.961148 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f47863-ff05-4755-afb5-b6aa45736a9c service swift] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:31.966274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55e48d-e10f-4b47-85f4-12c74d2fd881 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef518c9430e4274b1066241f821f4a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:31.967021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55e48d-e10f-4b47-85f4-12c74d2fd881 service swift] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:31.967374 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55e48d-e10f-4b47-85f4-12c74d2fd881 service swift] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:31.967716 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d55e48d-e10f-4b47-85f4-12c74d2fd881 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:31.978022 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d55e48d-e10f-4b47-85f4-12c74d2fd881 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:31.979712 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d55e48d-e10f-4b47-85f4-12c74d2fd881 service swift] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:31.988404 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1009/2043] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 23:02:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 23:02:32.222459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d06b3-2e51-4d50-941c-4ba1d847aea8 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:32.222790 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d06b3-2e51-4d50-941c-4ba1d847aea8 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:32.223071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d06b3-2e51-4d50-941c-4ba1d847aea8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:32.252116 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-7d1d06b3-2e51-4d50-941c-4ba1d847aea8 None None] MFA Rules not processed for user `faf778e713b54a638c1db4a5db6180b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:32.255080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7d1d06b3-2e51-4d50-941c-4ba1d847aea8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:32.270650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1035/2044] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 23:02:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:32.284179 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55e48d-e10f-4b47-85f4-12c74d2fd881 service swift] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:32.297504 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d55e48d-e10f-4b47-85f4-12c74d2fd881 service swift] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:32.305943 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8032c22-21dd-4586-96e0-a9275dea86d1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef518c9430e4274b1066241f821f4a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:32.306958 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8032c22-21dd-4586-96e0-a9275dea86d1 service swift] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:32.307240 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8032c22-21dd-4586-96e0-a9275dea86d1 service swift] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:32.307709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8032c22-21dd-4586-96e0-a9275dea86d1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:32.318095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8032c22-21dd-4586-96e0-a9275dea86d1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:32.320492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8032c22-21dd-4586-96e0-a9275dea86d1 service swift] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:32.331332 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1010/2045] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 23:02:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 23:02:32.717703 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29daf2-6f53-495f-a737-2ae9b19a2153 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:32.718158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29daf2-6f53-495f-a737-2ae9b19a2153 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:32.718511 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db29daf2-6f53-495f-a737-2ae9b19a2153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:32.737778 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-db29daf2-6f53-495f-a737-2ae9b19a2153 None None] MFA Rules not processed for user `faf778e713b54a638c1db4a5db6180b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:32.739324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-db29daf2-6f53-495f-a737-2ae9b19a2153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:32.746693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1036/2046] 10.222.0.17 () {58 vars in 967 bytes} [Thu Jun 16 23:02:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:32.758548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8032c22-21dd-4586-96e0-a9275dea86d1 service swift] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:32.768732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8032c22-21dd-4586-96e0-a9275dea86d1 service swift] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:32.773754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9994ee-4f7c-43f6-bda3-11463ab684bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef518c9430e4274b1066241f821f4a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:32.774396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9994ee-4f7c-43f6-bda3-11463ab684bd service swift] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:32.775245 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9994ee-4f7c-43f6-bda3-11463ab684bd service swift] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:32.775526 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9994ee-4f7c-43f6-bda3-11463ab684bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:32.784394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9994ee-4f7c-43f6-bda3-11463ab684bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:32.786050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9994ee-4f7c-43f6-bda3-11463ab684bd service swift] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:32.795536 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1011/2047] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 23:02:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 23:02:32.814428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f8076-d9c4-4974-a8b4-21f9a0c5c13b None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:32.814723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f8076-d9c4-4974-a8b4-21f9a0c5c13b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:32.814987 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583f8076-d9c4-4974-a8b4-21f9a0c5c13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:32.833016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-583f8076-d9c4-4974-a8b4-21f9a0c5c13b None None] MFA Rules not processed for user `243e592fbf1844d89f55dd5035a684f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:32.834227 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-583f8076-d9c4-4974-a8b4-21f9a0c5c13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:32.894452 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1037/2048] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:32 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:33.283438 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9994ee-4f7c-43f6-bda3-11463ab684bd service swift] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:33.357378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9994ee-4f7c-43f6-bda3-11463ab684bd service swift] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:33.362460 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61c8ca-99b0-4e54-8474-fad40ed71ac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:33.363439 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61c8ca-99b0-4e54-8474-fad40ed71ac6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:33.363967 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61c8ca-99b0-4e54-8474-fad40ed71ac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:33.364334 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61c8ca-99b0-4e54-8474-fad40ed71ac6 admin admin] PATH_INFO: `/v3/users/95029fe88d4141138ea871ef340631a4` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:33.365606 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61c8ca-99b0-4e54-8474-fad40ed71ac6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95029fe88d4141138ea871ef340631a4)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:33.367661 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d61c8ca-99b0-4e54-8474-fad40ed71ac6 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:33.448532 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2d61c8ca-99b0-4e54-8474-fad40ed71ac6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95029fe88d4141138ea871ef340631a4'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:33.449252 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2d61c8ca-99b0-4e54-8474-fad40ed71ac6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95029fe88d4141138ea871ef340631a4'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:33.455231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:33.464640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2d61c8ca-99b0-4e54-8474-fad40ed71ac6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95029fe88d4141138ea871ef340631a4'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:33.466572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2d61c8ca-99b0-4e54-8474-fad40ed71ac6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95029fe88d4141138ea871ef340631a4'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:33.481258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2d61c8ca-99b0-4e54-8474-fad40ed71ac6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95029fe88d4141138ea871ef340631a4'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:33.499252 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1012/2049] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:33 2022] DELETE /identity/v3/users/95029fe88d4141138ea871ef340631a4 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:33.503916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3edbcb-edb3-44bf-aaa1-ddca2323d8b9 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:33.504464 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3edbcb-edb3-44bf-aaa1-ddca2323d8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:33.504464 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3edbcb-edb3-44bf-aaa1-ddca2323d8b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:33.525604 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-df3edbcb-edb3-44bf-aaa1-ddca2323d8b9 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:33.527157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-df3edbcb-edb3-44bf-aaa1-ddca2323d8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:33.613670 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1038/2050] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:33.617369 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61c8ca-99b0-4e54-8474-fad40ed71ac6 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:33.618816 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-2d61c8ca-99b0-4e54-8474-fad40ed71ac6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:33.697305 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61c8ca-99b0-4e54-8474-fad40ed71ac6 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:33.701340 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64475c-d646-42eb-b4c8-cd21888b9180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:33.702102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64475c-d646-42eb-b4c8-cd21888b9180 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:33.702393 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64475c-d646-42eb-b4c8-cd21888b9180 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:33.702702 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d64475c-d646-42eb-b4c8-cd21888b9180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:33.714138 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64475c-d646-42eb-b4c8-cd21888b9180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:33.715845 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d64475c-d646-42eb-b4c8-cd21888b9180 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:33.728847 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1013/2051] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:34.075619 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3edbcb-edb3-44bf-aaa1-ddca2323d8b9 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:34.077672 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-df3edbcb-edb3-44bf-aaa1-ddca2323d8b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:34.090505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3edbcb-edb3-44bf-aaa1-ddca2323d8b9 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:34.095747 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff6d83-7b5d-4bf6-9d53-85b62e722efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:34.096730 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff6d83-7b5d-4bf6-9d53-85b62e722efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:34.097081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff6d83-7b5d-4bf6-9d53-85b62e722efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:34.097586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ff6d83-7b5d-4bf6-9d53-85b62e722efb admin admin] PATH_INFO: `/v3/projects/850f9eb2a4b442d8ae329065e0e30a27` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:34.099062 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff6d83-7b5d-4bf6-9d53-85b62e722efb admin admin] RBAC: Authorizing `identity:delete_project(project_id=850f9eb2a4b442d8ae329065e0e30a27)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:34.101393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ff6d83-7b5d-4bf6-9d53-85b62e722efb admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:34.133950 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:34.152976 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-f9ff6d83-7b5d-4bf6-9d53-85b62e722efb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '850f9eb2a4b442d8ae329065e0e30a27', 'request_id': 'req-f9ff6d83-7b5d-4bf6-9d53-85b62e722efb'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:34.161991 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-f9ff6d83-7b5d-4bf6-9d53-85b62e722efb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '850f9eb2a4b442d8ae329065e0e30a27', 'request_id': 'req-f9ff6d83-7b5d-4bf6-9d53-85b62e722efb'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:34.169133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-f9ff6d83-7b5d-4bf6-9d53-85b62e722efb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '850f9eb2a4b442d8ae329065e0e30a27', 'request_id': 'req-f9ff6d83-7b5d-4bf6-9d53-85b62e722efb'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:34.188934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-f9ff6d83-7b5d-4bf6-9d53-85b62e722efb admin admin] The token cache is being invalidate because project 850f9eb2a4b442d8ae329065e0e30a27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:34.189151 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-f9ff6d83-7b5d-4bf6-9d53-85b62e722efb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:34.189760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-f9ff6d83-7b5d-4bf6-9d53-85b62e722efb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:34.191492 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1039/2052] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:34 2022] DELETE /identity/v3/projects/850f9eb2a4b442d8ae329065e0e30a27 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:34.195528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64475c-d646-42eb-b4c8-cd21888b9180 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:34.207417 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d64475c-d646-42eb-b4c8-cd21888b9180 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:34.303001 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b11ed6b-9c7a-47b3-a170-e265a4a1a516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:34.303918 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11ed6b-9c7a-47b3-a170-e265a4a1a516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:34.304207 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11ed6b-9c7a-47b3-a170-e265a4a1a516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:34.304488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b11ed6b-9c7a-47b3-a170-e265a4a1a516 admin admin] PATH_INFO: `/v3/users/109311beeb944f36a72df99eff2de8b1` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:34.306223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11ed6b-9c7a-47b3-a170-e265a4a1a516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=109311beeb944f36a72df99eff2de8b1)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:34.308295 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b11ed6b-9c7a-47b3-a170-e265a4a1a516 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:34.358523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1b11ed6b-9c7a-47b3-a170-e265a4a1a516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '109311beeb944f36a72df99eff2de8b1'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:34.359574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1b11ed6b-9c7a-47b3-a170-e265a4a1a516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '109311beeb944f36a72df99eff2de8b1'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:34.367062 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1b11ed6b-9c7a-47b3-a170-e265a4a1a516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '109311beeb944f36a72df99eff2de8b1'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:34.367801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1b11ed6b-9c7a-47b3-a170-e265a4a1a516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '109311beeb944f36a72df99eff2de8b1'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:34.384925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1b11ed6b-9c7a-47b3-a170-e265a4a1a516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '109311beeb944f36a72df99eff2de8b1'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:34.386752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:34.403987 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1014/2053] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:34 2022] DELETE /identity/v3/users/109311beeb944f36a72df99eff2de8b1 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:34.693433 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff6d83-7b5d-4bf6-9d53-85b62e722efb admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:34.694913 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-f9ff6d83-7b5d-4bf6-9d53-85b62e722efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:34.710576 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ff6d83-7b5d-4bf6-9d53-85b62e722efb admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:34.768379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7791b5-64fb-44dd-b5d5-636c7cdbbcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:34.769242 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7791b5-64fb-44dd-b5d5-636c7cdbbcd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:34.769529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7791b5-64fb-44dd-b5d5-636c7cdbbcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:34.769807 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7791b5-64fb-44dd-b5d5-636c7cdbbcd9 admin admin] PATH_INFO: `/v3/projects/699bffbce44b4a1586e722fc867b6e2e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:34.771011 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7791b5-64fb-44dd-b5d5-636c7cdbbcd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=699bffbce44b4a1586e722fc867b6e2e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:34.772920 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7791b5-64fb-44dd-b5d5-636c7cdbbcd9 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:34.827186 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bc7791b5-64fb-44dd-b5d5-636c7cdbbcd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '699bffbce44b4a1586e722fc867b6e2e', 'request_id': 'req-bc7791b5-64fb-44dd-b5d5-636c7cdbbcd9'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:34.835321 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bc7791b5-64fb-44dd-b5d5-636c7cdbbcd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '699bffbce44b4a1586e722fc867b6e2e', 'request_id': 'req-bc7791b5-64fb-44dd-b5d5-636c7cdbbcd9'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:34.841757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bc7791b5-64fb-44dd-b5d5-636c7cdbbcd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '699bffbce44b4a1586e722fc867b6e2e', 'request_id': 'req-bc7791b5-64fb-44dd-b5d5-636c7cdbbcd9'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:34.859145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bc7791b5-64fb-44dd-b5d5-636c7cdbbcd9 admin admin] The token cache is being invalidate because project 699bffbce44b4a1586e722fc867b6e2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:34.859442 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bc7791b5-64fb-44dd-b5d5-636c7cdbbcd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:34.860101 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bc7791b5-64fb-44dd-b5d5-636c7cdbbcd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:34.861862 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1040/2054] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:34 2022] DELETE /identity/v3/projects/699bffbce44b4a1586e722fc867b6e2e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:34.866561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98276c2e-ebfb-4b04-8017-daf22f4bb428 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:34.866986 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98276c2e-ebfb-4b04-8017-daf22f4bb428 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:34.867180 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98276c2e-ebfb-4b04-8017-daf22f4bb428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:34.887604 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-98276c2e-ebfb-4b04-8017-daf22f4bb428 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:34.889182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-98276c2e-ebfb-4b04-8017-daf22f4bb428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:34.967357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1015/2055] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:34.973731 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441ffca-521c-40ed-b4af-f9d33806ea44 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:34.974027 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441ffca-521c-40ed-b4af-f9d33806ea44 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:34.974315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b441ffca-521c-40ed-b4af-f9d33806ea44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:34.995808 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-b441ffca-521c-40ed-b4af-f9d33806ea44 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:34.997318 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b441ffca-521c-40ed-b4af-f9d33806ea44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:35.011521 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1041/2056] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:35.016613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98276c2e-ebfb-4b04-8017-daf22f4bb428 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:35.029991 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98276c2e-ebfb-4b04-8017-daf22f4bb428 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:35.035893 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b441ffca-521c-40ed-b4af-f9d33806ea44 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:35.037462 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b441ffca-521c-40ed-b4af-f9d33806ea44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:35.040894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a852805-5e6d-4674-a2ea-9c9406382e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:35.043176 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a852805-5e6d-4674-a2ea-9c9406382e2c admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:35.043931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a852805-5e6d-4674-a2ea-9c9406382e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:35.044628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a852805-5e6d-4674-a2ea-9c9406382e2c admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:35.046383 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a852805-5e6d-4674-a2ea-9c9406382e2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:02:35.048072 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a852805-5e6d-4674-a2ea-9c9406382e2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:35.051902 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b441ffca-521c-40ed-b4af-f9d33806ea44 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:35.053571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a852805-5e6d-4674-a2ea-9c9406382e2c admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:35.057598 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e3b64f-07e9-4603-9221-d20485eec251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:35.058777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3b64f-07e9-4603-9221-d20485eec251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:35.059220 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3b64f-07e9-4603-9221-d20485eec251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:35.059607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e3b64f-07e9-4603-9221-d20485eec251 admin admin] PATH_INFO: `/v3/users/0ee1c54733f648ada66a143ce7516375` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:35.061150 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3b64f-07e9-4603-9221-d20485eec251 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ee1c54733f648ada66a143ce7516375)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:35.063286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e3b64f-07e9-4603-9221-d20485eec251 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:35.071084 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1016/2057] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:02:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:02:35.075476 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a852805-5e6d-4674-a2ea-9c9406382e2c admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:35.087822 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a852805-5e6d-4674-a2ea-9c9406382e2c admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:35.093132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fedc9-c41f-4c85-8a90-e930cd62d3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:35.093799 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fedc9-c41f-4c85-8a90-e930cd62d3c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:35.094100 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fedc9-c41f-4c85-8a90-e930cd62d3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:35.094366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43fedc9-c41f-4c85-8a90-e930cd62d3c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:35.095089 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43fedc9-c41f-4c85-8a90-e930cd62d3c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:35.097076 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43fedc9-c41f-4c85-8a90-e930cd62d3c7 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:35.112002 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:35.132563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-49e3b64f-07e9-4603-9221-d20485eec251 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ee1c54733f648ada66a143ce7516375'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:35.144413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1017/2058] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:35 2022] POST /identity/v3/projects => generated 365 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:35.144587 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-49e3b64f-07e9-4603-9221-d20485eec251 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ee1c54733f648ada66a143ce7516375'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:35.145406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-49e3b64f-07e9-4603-9221-d20485eec251 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ee1c54733f648ada66a143ce7516375'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:35.145524 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fedc9-c41f-4c85-8a90-e930cd62d3c7 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:35.161783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-49e3b64f-07e9-4603-9221-d20485eec251 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ee1c54733f648ada66a143ce7516375'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:35.162514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-49e3b64f-07e9-4603-9221-d20485eec251 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ee1c54733f648ada66a143ce7516375'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:35.176438 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1042/2059] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:35 2022] DELETE /identity/v3/users/0ee1c54733f648ada66a143ce7516375 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:02:35.180765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988da638-0f75-47aa-a496-e16380b892e4 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:35.181064 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988da638-0f75-47aa-a496-e16380b892e4 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:35.181350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988da638-0f75-47aa-a496-e16380b892e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:35.199111 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-988da638-0f75-47aa-a496-e16380b892e4 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:35.200436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-988da638-0f75-47aa-a496-e16380b892e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:35.210393 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43fedc9-c41f-4c85-8a90-e930cd62d3c7 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:35.267355 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1043/2060] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:35.269625 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e3591-eed0-40b5-9055-60913b67d6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:35.270324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e3591-eed0-40b5-9055-60913b67d6ab admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:35.270634 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e3591-eed0-40b5-9055-60913b67d6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:35.270909 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e3591-eed0-40b5-9055-60913b67d6ab admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:35.271599 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e3591-eed0-40b5-9055-60913b67d6ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:35.273630 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e3591-eed0-40b5-9055-60913b67d6ab admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:35.275029 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988da638-0f75-47aa-a496-e16380b892e4 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:35.276393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-988da638-0f75-47aa-a496-e16380b892e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:35.282860 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:35.302894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1018/2061] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:35 2022] POST /identity/v3/users => generated 283 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:35.304065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e3591-eed0-40b5-9055-60913b67d6ab admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:35.306310 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-4e8e3591-eed0-40b5-9055-60913b67d6ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:35.319927 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e3591-eed0-40b5-9055-60913b67d6ab admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:35.325972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8d06a-1722-4e97-8d81-ce934f4d1909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:35.326683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8d06a-1722-4e97-8d81-ce934f4d1909 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:35.326972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8d06a-1722-4e97-8d81-ce934f4d1909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:35.327442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e8d06a-1722-4e97-8d81-ce934f4d1909 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:35.328333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8d06a-1722-4e97-8d81-ce934f4d1909 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:35.329948 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e8d06a-1722-4e97-8d81-ce934f4d1909 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:35.341842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1019/2062] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:35.342528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988da638-0f75-47aa-a496-e16380b892e4 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:35.346491 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8d06a-1722-4e97-8d81-ce934f4d1909 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:35.348086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412367b-888b-489c-8a02-112f1c9123f9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:35.348780 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412367b-888b-489c-8a02-112f1c9123f9 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:35.349069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412367b-888b-489c-8a02-112f1c9123f9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:35.349355 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412367b-888b-489c-8a02-112f1c9123f9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:35.359155 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e8d06a-1722-4e97-8d81-ce934f4d1909 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:35.360784 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412367b-888b-489c-8a02-112f1c9123f9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:35.362123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f412367b-888b-489c-8a02-112f1c9123f9 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:35.364762 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a298117e-5606-4b4e-87b1-f88996229e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:35.365789 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298117e-5606-4b4e-87b1-f88996229e4b admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:35.365789 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298117e-5606-4b4e-87b1-f88996229e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:35.366221 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a298117e-5606-4b4e-87b1-f88996229e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:35.366989 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298117e-5606-4b4e-87b1-f88996229e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:35.369685 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a298117e-5606-4b4e-87b1-f88996229e4b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:35.372939 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1044/2063] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:35.382731 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1020/2064] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:35.387152 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412367b-888b-489c-8a02-112f1c9123f9 service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:35.400434 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412367b-888b-489c-8a02-112f1c9123f9 service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:35.406313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29aed7-7886-4ea3-82be-809354d68ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:35.407375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29aed7-7886-4ea3-82be-809354d68ebd admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:35.407663 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29aed7-7886-4ea3-82be-809354d68ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:35.407941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c29aed7-7886-4ea3-82be-809354d68ebd admin admin] PATH_INFO: `/v3/projects/a277d0f4925c4e43bf58cb7d8e0465e9/users/5e998d8a00e942818aaa93296f828219/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:35.424679 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29aed7-7886-4ea3-82be-809354d68ebd admin admin] RBAC: Authorizing `identity:create_grant(project_id=a277d0f4925c4e43bf58cb7d8e0465e9, user_id=5e998d8a00e942818aaa93296f828219, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:35.428469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c29aed7-7886-4ea3-82be-809354d68ebd admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:35.445014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1045/2065] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:35 2022] PUT /identity/v3/projects/a277d0f4925c4e43bf58cb7d8e0465e9/users/5e998d8a00e942818aaa93296f828219/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:35.459985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74163fa-4b99-4139-8709-1d213e56ef1e None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:35.460293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74163fa-4b99-4139-8709-1d213e56ef1e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:35.460892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74163fa-4b99-4139-8709-1d213e56ef1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:35.491524 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-c74163fa-4b99-4139-8709-1d213e56ef1e None None] MFA Rules not processed for user `5e998d8a00e942818aaa93296f828219`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:35.492722 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c74163fa-4b99-4139-8709-1d213e56ef1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:35.588239 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1021/2066] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:35 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:35.593203 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29aed7-7886-4ea3-82be-809354d68ebd admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:35.629093 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:35.668253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c29aed7-7886-4ea3-82be-809354d68ebd admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:35.683102 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb1a14-d9fa-45b1-ac83-9aaa9c0f238e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:35.683102 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb1a14-d9fa-45b1-ac83-9aaa9c0f238e admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:35.683102 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb1a14-d9fa-45b1-ac83-9aaa9c0f238e admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:35.683102 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fb1a14-d9fa-45b1-ac83-9aaa9c0f238e admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:35.683434 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb1a14-d9fa-45b1-ac83-9aaa9c0f238e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:35.683516 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74163fa-4b99-4139-8709-1d213e56ef1e None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:35.685255 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c74163fa-4b99-4139-8709-1d213e56ef1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:35.685372 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fb1a14-d9fa-45b1-ac83-9aaa9c0f238e admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:35.702327 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74163fa-4b99-4139-8709-1d213e56ef1e None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:35.709539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5c865a-5076-43bc-a03a-e375f2046b89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:35.710648 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c865a-5076-43bc-a03a-e375f2046b89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:35.710782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c865a-5076-43bc-a03a-e375f2046b89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:35.711006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5c865a-5076-43bc-a03a-e375f2046b89 admin admin] PATH_INFO: `/v3/projects/c90a3d32fe204da7a969528722141e75` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:35.712812 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c865a-5076-43bc-a03a-e375f2046b89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c90a3d32fe204da7a969528722141e75)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:35.714982 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5c865a-5076-43bc-a03a-e375f2046b89 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:35.722998 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1046/2067] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:35 2022] POST /identity/v3/projects => generated 363 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:35.724028 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb1a14-d9fa-45b1-ac83-9aaa9c0f238e admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:35.786938 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-ec5c865a-5076-43bc-a03a-e375f2046b89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c90a3d32fe204da7a969528722141e75', 'request_id': 'req-ec5c865a-5076-43bc-a03a-e375f2046b89'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:35.796491 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-ec5c865a-5076-43bc-a03a-e375f2046b89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c90a3d32fe204da7a969528722141e75', 'request_id': 'req-ec5c865a-5076-43bc-a03a-e375f2046b89'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:35.804889 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-ec5c865a-5076-43bc-a03a-e375f2046b89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c90a3d32fe204da7a969528722141e75', 'request_id': 'req-ec5c865a-5076-43bc-a03a-e375f2046b89'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:35.814531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fb1a14-d9fa-45b1-ac83-9aaa9c0f238e admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:35.821722 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-ec5c865a-5076-43bc-a03a-e375f2046b89 admin admin] The token cache is being invalidate because project c90a3d32fe204da7a969528722141e75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:35.822731 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-ec5c865a-5076-43bc-a03a-e375f2046b89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:35.824172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-ec5c865a-5076-43bc-a03a-e375f2046b89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:35.826671 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1022/2068] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:35 2022] DELETE /identity/v3/projects/c90a3d32fe204da7a969528722141e75 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:35.832843 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d035e-f11f-4e87-a6ea-d7268d441bcd None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:35.833869 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d035e-f11f-4e87-a6ea-d7268d441bcd None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:35.834577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d035e-f11f-4e87-a6ea-d7268d441bcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:35.864954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-285d035e-f11f-4e87-a6ea-d7268d441bcd None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:35.866319 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-285d035e-f11f-4e87-a6ea-d7268d441bcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:35.889674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09894884-b473-4757-9532-310a05c60983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:35.890464 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09894884-b473-4757-9532-310a05c60983 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:35.891347 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09894884-b473-4757-9532-310a05c60983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:35.891347 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09894884-b473-4757-9532-310a05c60983 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:35.892141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09894884-b473-4757-9532-310a05c60983 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:35.894342 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09894884-b473-4757-9532-310a05c60983 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:35.929663 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1047/2069] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:35 2022] POST /identity/v3/users => generated 288 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:35.931338 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09894884-b473-4757-9532-310a05c60983 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:35.933797 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-09894884-b473-4757-9532-310a05c60983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:35.938074 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:35.962858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09894884-b473-4757-9532-310a05c60983 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:35.966619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1023/2070] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:35.970463 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df71c2-4eb5-4afc-b3f1-4d6e33e0ff4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:35.971202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df71c2-4eb5-4afc-b3f1-4d6e33e0ff4f admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:35.971491 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df71c2-4eb5-4afc-b3f1-4d6e33e0ff4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:35.971777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73df71c2-4eb5-4afc-b3f1-4d6e33e0ff4f admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:35.972787 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df71c2-4eb5-4afc-b3f1-4d6e33e0ff4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:35.974425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73df71c2-4eb5-4afc-b3f1-4d6e33e0ff4f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:35.975743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005a200-8f50-4474-aa2f-e6155eeec7bc None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:35.976142 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005a200-8f50-4474-aa2f-e6155eeec7bc None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:35.976516 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6005a200-8f50-4474-aa2f-e6155eeec7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:35.988650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1048/2071] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:35.991750 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df71c2-4eb5-4afc-b3f1-4d6e33e0ff4f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:36.004238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-6005a200-8f50-4474-aa2f-e6155eeec7bc None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:36.005238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73df71c2-4eb5-4afc-b3f1-4d6e33e0ff4f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:36.005388 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6005a200-8f50-4474-aa2f-e6155eeec7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:36.011359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2b377-6b76-44fc-b00c-e60a350b516c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:36.012422 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2b377-6b76-44fc-b00c-e60a350b516c admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:36.012826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2b377-6b76-44fc-b00c-e60a350b516c admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:36.013201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab2b377-6b76-44fc-b00c-e60a350b516c admin admin] PATH_INFO: `/v3/projects/25b953f244224f3dbbe2e93caa096bf6/users/7a68902bb9f64c998903fe36cbc7c607/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:36.015760 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1024/2072] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:36.019824 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005a200-8f50-4474-aa2f-e6155eeec7bc None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:36.030612 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6005a200-8f50-4474-aa2f-e6155eeec7bc None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:36.036889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab2b377-6b76-44fc-b00c-e60a350b516c admin admin] RBAC: Authorizing `identity:create_grant(project_id=25b953f244224f3dbbe2e93caa096bf6, user_id=7a68902bb9f64c998903fe36cbc7c607, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:36.040092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab2b377-6b76-44fc-b00c-e60a350b516c admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:36.041244 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468e266-af9e-497c-9b4b-068ea0f8fa6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:36.042095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468e266-af9e-497c-9b4b-068ea0f8fa6d admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:36.042796 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468e266-af9e-497c-9b4b-068ea0f8fa6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:36.043276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c468e266-af9e-497c-9b4b-068ea0f8fa6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:36.044829 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468e266-af9e-497c-9b4b-068ea0f8fa6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:02:36.045600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468e266-af9e-497c-9b4b-068ea0f8fa6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:36.048238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c468e266-af9e-497c-9b4b-068ea0f8fa6d admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:36.058174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1049/2073] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:35 2022] PUT /identity/v3/projects/25b953f244224f3dbbe2e93caa096bf6/users/7a68902bb9f64c998903fe36cbc7c607/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:36.061937 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2b377-6b76-44fc-b00c-e60a350b516c admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:36.070536 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1025/2074] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:02:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:02:36.079311 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468e266-af9e-497c-9b4b-068ea0f8fa6d admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:36.091593 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c468e266-af9e-497c-9b4b-068ea0f8fa6d admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:36.100492 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:36.160917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab2b377-6b76-44fc-b00c-e60a350b516c admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:36.165923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4283245-5b50-4dcd-86e4-ea8d4ba4d43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:36.166310 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659ca84-5436-46d6-9aa4-7a8323df8f3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:36.166604 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4283245-5b50-4dcd-86e4-ea8d4ba4d43e admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:36.166899 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4283245-5b50-4dcd-86e4-ea8d4ba4d43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:36.167039 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659ca84-5436-46d6-9aa4-7a8323df8f3a admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:36.167195 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4283245-5b50-4dcd-86e4-ea8d4ba4d43e admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:36.167330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659ca84-5436-46d6-9aa4-7a8323df8f3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:36.167662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659ca84-5436-46d6-9aa4-7a8323df8f3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:36.167851 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4283245-5b50-4dcd-86e4-ea8d4ba4d43e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:36.168421 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659ca84-5436-46d6-9aa4-7a8323df8f3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:36.169736 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4283245-5b50-4dcd-86e4-ea8d4ba4d43e admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:36.170005 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659ca84-5436-46d6-9aa4-7a8323df8f3a admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:36.180469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1050/2075] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:02:36.185339 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659ca84-5436-46d6-9aa4-7a8323df8f3a admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:36.195456 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659ca84-5436-46d6-9aa4-7a8323df8f3a admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:36.198221 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1026/2076] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:36 2022] POST /identity/v3/projects => generated 397 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:36.200328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16652a64-b154-4983-b21e-a2906764309a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:36.201366 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16652a64-b154-4983-b21e-a2906764309a admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:36.201657 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16652a64-b154-4983-b21e-a2906764309a admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:36.201936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16652a64-b154-4983-b21e-a2906764309a admin admin] PATH_INFO: `/v3/projects/25b953f244224f3dbbe2e93caa096bf6/users/7a68902bb9f64c998903fe36cbc7c607/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:36.202359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4283245-5b50-4dcd-86e4-ea8d4ba4d43e admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:36.204060 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16652a64-b154-4983-b21e-a2906764309a admin admin] RBAC: Authorizing `identity:create_grant(project_id=25b953f244224f3dbbe2e93caa096bf6, user_id=7a68902bb9f64c998903fe36cbc7c607, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:36.206754 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16652a64-b154-4983-b21e-a2906764309a admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:36.224672 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1051/2077] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:36 2022] PUT /identity/v3/projects/25b953f244224f3dbbe2e93caa096bf6/users/7a68902bb9f64c998903fe36cbc7c607/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:02:36.236133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd75d6-37b5-4983-88e1-7a9eb26b0673 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:36.236429 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd75d6-37b5-4983-88e1-7a9eb26b0673 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:36.236713 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bd75d6-37b5-4983-88e1-7a9eb26b0673 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:36.268527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-18bd75d6-37b5-4983-88e1-7a9eb26b0673 None None] MFA Rules not processed for user `7a68902bb9f64c998903fe36cbc7c607`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:36.269900 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-18bd75d6-37b5-4983-88e1-7a9eb26b0673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:36.283454 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4283245-5b50-4dcd-86e4-ea8d4ba4d43e admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:36.335245 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:36.342942 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1052/2078] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:36 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:36.343093 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9abd3e-9e73-49c0-bd45-79fb79e918fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:36.343663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9abd3e-9e73-49c0-bd45-79fb79e918fd admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:36.344284 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9abd3e-9e73-49c0-bd45-79fb79e918fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:36.344571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee9abd3e-9e73-49c0-bd45-79fb79e918fd admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:36.345297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9abd3e-9e73-49c0-bd45-79fb79e918fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:36.347430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee9abd3e-9e73-49c0-bd45-79fb79e918fd admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:36.355514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd75d6-37b5-4983-88e1-7a9eb26b0673 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:36.357303 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-18bd75d6-37b5-4983-88e1-7a9eb26b0673 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:36.369204 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1027/2079] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:36 2022] POST /identity/v3/users => generated 299 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:36.373181 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9abd3e-9e73-49c0-bd45-79fb79e918fd admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:36.383002 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee9abd3e-9e73-49c0-bd45-79fb79e918fd admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:36.387908 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedc24f-fec2-4505-a377-6a2159b4af46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:36.388555 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedc24f-fec2-4505-a377-6a2159b4af46 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:36.388850 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedc24f-fec2-4505-a377-6a2159b4af46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:36.389135 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bedc24f-fec2-4505-a377-6a2159b4af46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:36.389865 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bedc24f-fec2-4505-a377-6a2159b4af46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:36.391591 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bedc24f-fec2-4505-a377-6a2159b4af46 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:36.402165 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1028/2080] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:02:36.405901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedc24f-fec2-4505-a377-6a2159b4af46 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:36.415235 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bedc24f-fec2-4505-a377-6a2159b4af46 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:36.415775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bd75d6-37b5-4983-88e1-7a9eb26b0673 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:36.419488 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c5b13-c244-4968-8011-f65c6f9fa7fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:36.420143 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c5b13-c244-4968-8011-f65c6f9fa7fb service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:36.420258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21822af-d3e0-45c4-871e-7eabf62490da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:36.420433 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c5b13-c244-4968-8011-f65c6f9fa7fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:36.420718 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c5b13-c244-4968-8011-f65c6f9fa7fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:36.421213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21822af-d3e0-45c4-871e-7eabf62490da admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:36.421595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21822af-d3e0-45c4-871e-7eabf62490da admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:36.421974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21822af-d3e0-45c4-871e-7eabf62490da admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:36.422817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21822af-d3e0-45c4-871e-7eabf62490da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:36.424399 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21822af-d3e0-45c4-871e-7eabf62490da admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:36.430195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c5b13-c244-4968-8011-f65c6f9fa7fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:36.432200 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c5b13-c244-4968-8011-f65c6f9fa7fb service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:36.435615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1029/2081] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:36.439466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21822af-d3e0-45c4-871e-7eabf62490da admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:36.442020 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1053/2082] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:36 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:02:36.449124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21822af-d3e0-45c4-871e-7eabf62490da admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:36.453859 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433bae9-ae67-46e6-86bf-b386e610f0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:36.454757 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433bae9-ae67-46e6-86bf-b386e610f0ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:36.455044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433bae9-ae67-46e6-86bf-b386e610f0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:36.455322 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1433bae9-ae67-46e6-86bf-b386e610f0ea admin admin] PATH_INFO: `/v3/projects/71182a78451c403fb266ac5577f00a40/users/cca3974ca425405caa7bd3dd7b5b3555/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:36.474154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433bae9-ae67-46e6-86bf-b386e610f0ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=71182a78451c403fb266ac5577f00a40, user_id=cca3974ca425405caa7bd3dd7b5b3555, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:36.475639 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1433bae9-ae67-46e6-86bf-b386e610f0ea admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:36.489807 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1030/2083] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:36 2022] PUT /identity/v3/projects/71182a78451c403fb266ac5577f00a40/users/cca3974ca425405caa7bd3dd7b5b3555/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:02:36.500248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e2605-4efa-4036-9167-f038492c5df7 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:36.500675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e2605-4efa-4036-9167-f038492c5df7 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:36.501045 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36e2605-4efa-4036-9167-f038492c5df7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:36.512953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433bae9-ae67-46e6-86bf-b386e610f0ea admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:36.530307 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a36e2605-4efa-4036-9167-f038492c5df7 None None] MFA Rules not processed for user `cca3974ca425405caa7bd3dd7b5b3555`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:36.531568 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a36e2605-4efa-4036-9167-f038492c5df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:36.545135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:36.569033 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1433bae9-ae67-46e6-86bf-b386e610f0ea admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:36.572870 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4d7cb-ba02-4c06-8979-6d82b66a9179 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:36.573528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4d7cb-ba02-4c06-8979-6d82b66a9179 service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:36.573818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4d7cb-ba02-4c06-8979-6d82b66a9179 service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:36.574098 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4d7cb-ba02-4c06-8979-6d82b66a9179 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:36.575945 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-04e4d7cb-ba02-4c06-8979-6d82b66a9179 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:36.618737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1054/2084] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:36 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:36.641306 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e2605-4efa-4036-9167-f038492c5df7 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:36.643064 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a36e2605-4efa-4036-9167-f038492c5df7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:36.656355 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36e2605-4efa-4036-9167-f038492c5df7 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:36.660855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b7c472-12e7-4bd5-8b8a-915976aa4667 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:36.660995 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4d7cb-ba02-4c06-8979-6d82b66a9179 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:36.661654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7c472-12e7-4bd5-8b8a-915976aa4667 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:36.662122 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7c472-12e7-4bd5-8b8a-915976aa4667 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:36.662507 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b7c472-12e7-4bd5-8b8a-915976aa4667 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:36.662670 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4d7cb-ba02-4c06-8979-6d82b66a9179 service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:36.672169 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b7c472-12e7-4bd5-8b8a-915976aa4667 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:36.672424 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1031/2085] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:36 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:02:36.674173 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b7c472-12e7-4bd5-8b8a-915976aa4667 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:36.691895 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1055/2086] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:36 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:38.298565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4d7cb-ba02-4c06-8979-6d82b66a9179 service cinder] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:38.300223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-04e4d7cb-ba02-4c06-8979-6d82b66a9179 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:38.362884 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:38.370801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4d7cb-ba02-4c06-8979-6d82b66a9179 service cinder] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:38.375927 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08528f7a-1e2c-4d7d-9eda-332c261f83f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:38.376807 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08528f7a-1e2c-4d7d-9eda-332c261f83f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:38.377097 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08528f7a-1e2c-4d7d-9eda-332c261f83f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:38.377377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08528f7a-1e2c-4d7d-9eda-332c261f83f9 admin admin] PATH_INFO: `/v3/users/910ef56632b3482699a33a6157c8d289` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:38.378578 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08528f7a-1e2c-4d7d-9eda-332c261f83f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=910ef56632b3482699a33a6157c8d289)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:38.380524 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08528f7a-1e2c-4d7d-9eda-332c261f83f9 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:38.429901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-08528f7a-1e2c-4d7d-9eda-332c261f83f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '910ef56632b3482699a33a6157c8d289'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:38.431107 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-08528f7a-1e2c-4d7d-9eda-332c261f83f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '910ef56632b3482699a33a6157c8d289'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:38.441678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-08528f7a-1e2c-4d7d-9eda-332c261f83f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '910ef56632b3482699a33a6157c8d289'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:38.442351 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-08528f7a-1e2c-4d7d-9eda-332c261f83f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '910ef56632b3482699a33a6157c8d289'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:38.453886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-08528f7a-1e2c-4d7d-9eda-332c261f83f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '910ef56632b3482699a33a6157c8d289'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:38.467517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1032/2087] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:38 2022] DELETE /identity/v3/users/910ef56632b3482699a33a6157c8d289 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:38.471941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c130a1d-2230-4f16-adfa-f39f614ff5a4 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:38.472427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c130a1d-2230-4f16-adfa-f39f614ff5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:38.472529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c130a1d-2230-4f16-adfa-f39f614ff5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:38.492438 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-8c130a1d-2230-4f16-adfa-f39f614ff5a4 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:38.494370 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-8c130a1d-2230-4f16-adfa-f39f614ff5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:38.576570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1056/2088] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:38.583446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08528f7a-1e2c-4d7d-9eda-332c261f83f9 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:38.585165 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-08528f7a-1e2c-4d7d-9eda-332c261f83f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:38.659153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08528f7a-1e2c-4d7d-9eda-332c261f83f9 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:38.663291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c2dd62-2524-430d-84e1-0f4995e4b053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:38.664022 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2dd62-2524-430d-84e1-0f4995e4b053 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:38.664435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2dd62-2524-430d-84e1-0f4995e4b053 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:38.664768 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c2dd62-2524-430d-84e1-0f4995e4b053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:38.676830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2dd62-2524-430d-84e1-0f4995e4b053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:38.678973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c2dd62-2524-430d-84e1-0f4995e4b053 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:38.697251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1033/2089] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:39.006444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c130a1d-2230-4f16-adfa-f39f614ff5a4 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:39.007889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-8c130a1d-2230-4f16-adfa-f39f614ff5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:39.020096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c130a1d-2230-4f16-adfa-f39f614ff5a4 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:39.024749 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da5742-cbd5-446d-8cd2-f26bd3cff7f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:39.025606 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da5742-cbd5-446d-8cd2-f26bd3cff7f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:39.025889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da5742-cbd5-446d-8cd2-f26bd3cff7f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:39.026162 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da5742-cbd5-446d-8cd2-f26bd3cff7f9 admin admin] PATH_INFO: `/v3/projects/192ad3f1e3c84e1b923d74cac0f62fbf` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:39.027384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da5742-cbd5-446d-8cd2-f26bd3cff7f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=192ad3f1e3c84e1b923d74cac0f62fbf)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:39.029378 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1da5742-cbd5-446d-8cd2-f26bd3cff7f9 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:39.077207 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a1da5742-cbd5-446d-8cd2-f26bd3cff7f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '192ad3f1e3c84e1b923d74cac0f62fbf', 'request_id': 'req-a1da5742-cbd5-446d-8cd2-f26bd3cff7f9'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:39.083801 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a1da5742-cbd5-446d-8cd2-f26bd3cff7f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '192ad3f1e3c84e1b923d74cac0f62fbf', 'request_id': 'req-a1da5742-cbd5-446d-8cd2-f26bd3cff7f9'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:39.090210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a1da5742-cbd5-446d-8cd2-f26bd3cff7f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '192ad3f1e3c84e1b923d74cac0f62fbf', 'request_id': 'req-a1da5742-cbd5-446d-8cd2-f26bd3cff7f9'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:39.104506 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a1da5742-cbd5-446d-8cd2-f26bd3cff7f9 admin admin] The token cache is being invalidate because project 192ad3f1e3c84e1b923d74cac0f62fbf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:39.104758 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a1da5742-cbd5-446d-8cd2-f26bd3cff7f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:39.105392 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a1da5742-cbd5-446d-8cd2-f26bd3cff7f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:39.106985 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1057/2090] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:39 2022] DELETE /identity/v3/projects/192ad3f1e3c84e1b923d74cac0f62fbf => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:39.113672 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c011e-7ec7-4246-9cba-b0c97e1259d5 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:39.113971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c011e-7ec7-4246-9cba-b0c97e1259d5 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:39.114252 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348c011e-7ec7-4246-9cba-b0c97e1259d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:39.132476 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-348c011e-7ec7-4246-9cba-b0c97e1259d5 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:39.133705 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-348c011e-7ec7-4246-9cba-b0c97e1259d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:39.193156 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1034/2091] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:39.201625 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452cd17-a6e1-42b1-8195-e3860a5539dc None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:39.201920 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452cd17-a6e1-42b1-8195-e3860a5539dc None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:39.202200 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452cd17-a6e1-42b1-8195-e3860a5539dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:39.223165 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-c452cd17-a6e1-42b1-8195-e3860a5539dc None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:39.224364 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c452cd17-a6e1-42b1-8195-e3860a5539dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:39.238517 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1058/2092] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:39.243383 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c011e-7ec7-4246-9cba-b0c97e1259d5 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:39.253276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348c011e-7ec7-4246-9cba-b0c97e1259d5 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:39.257949 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859250ac-18b6-4ec1-aa4b-d7d338b481da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:39.258633 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859250ac-18b6-4ec1-aa4b-d7d338b481da admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:39.258925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859250ac-18b6-4ec1-aa4b-d7d338b481da admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:39.259206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859250ac-18b6-4ec1-aa4b-d7d338b481da admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:39.259965 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859250ac-18b6-4ec1-aa4b-d7d338b481da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:02:39.260279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859250ac-18b6-4ec1-aa4b-d7d338b481da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:39.261883 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859250ac-18b6-4ec1-aa4b-d7d338b481da admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:39.274296 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1035/2093] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 23:02:39.278557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452cd17-a6e1-42b1-8195-e3860a5539dc None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:39.287726 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452cd17-a6e1-42b1-8195-e3860a5539dc None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:39.292414 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a848dc8-2c3f-405a-8619-e96728f0ec87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:39.293067 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a848dc8-2c3f-405a-8619-e96728f0ec87 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:39.293355 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a848dc8-2c3f-405a-8619-e96728f0ec87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:39.293634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a848dc8-2c3f-405a-8619-e96728f0ec87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:39.294331 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a848dc8-2c3f-405a-8619-e96728f0ec87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:39.296247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a848dc8-2c3f-405a-8619-e96728f0ec87 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:39.315932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:39.325144 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1059/2094] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:39 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:39.328876 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859250ac-18b6-4ec1-aa4b-d7d338b481da admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:39.335400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:39.339564 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859250ac-18b6-4ec1-aa4b-d7d338b481da admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:39.410752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e2d3c-0641-4371-9974-4d2eb65ff3da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:39.411496 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e2d3c-0641-4371-9974-4d2eb65ff3da admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:39.411783 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e2d3c-0641-4371-9974-4d2eb65ff3da admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:39.412058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e2d3c-0641-4371-9974-4d2eb65ff3da admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:39.412748 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e2d3c-0641-4371-9974-4d2eb65ff3da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:39.414695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e2d3c-0641-4371-9974-4d2eb65ff3da admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:39.437683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1036/2095] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:39 2022] POST /identity/v3/users => generated 292 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:02:39.440412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a848dc8-2c3f-405a-8619-e96728f0ec87 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:39.453121 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a848dc8-2c3f-405a-8619-e96728f0ec87 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:39.458400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd95bc5-4d21-4915-8c5f-a3b21f41fd66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:39.459364 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd95bc5-4d21-4915-8c5f-a3b21f41fd66 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:39.459656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd95bc5-4d21-4915-8c5f-a3b21f41fd66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:39.459945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd95bc5-4d21-4915-8c5f-a3b21f41fd66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:39.460783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd95bc5-4d21-4915-8c5f-a3b21f41fd66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:39.462875 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd95bc5-4d21-4915-8c5f-a3b21f41fd66 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:39.474503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1060/2096] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:39.479342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e2d3c-0641-4371-9974-4d2eb65ff3da admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:39.490460 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e2d3c-0641-4371-9974-4d2eb65ff3da admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:39.498707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ea324-85ad-44c3-a9b0-53230e3ceada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:39.499427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ea324-85ad-44c3-a9b0-53230e3ceada admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:39.499720 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ea324-85ad-44c3-a9b0-53230e3ceada admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:39.499999 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ea324-85ad-44c3-a9b0-53230e3ceada admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:39.500914 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ea324-85ad-44c3-a9b0-53230e3ceada admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:39.502768 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ea324-85ad-44c3-a9b0-53230e3ceada admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:39.515582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1037/2097] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:39.519882 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd95bc5-4d21-4915-8c5f-a3b21f41fd66 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:39.531156 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd95bc5-4d21-4915-8c5f-a3b21f41fd66 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:39.536847 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a70126-0f05-4f30-94a8-67745d9b42ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:39.538425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a70126-0f05-4f30-94a8-67745d9b42ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:39.538742 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a70126-0f05-4f30-94a8-67745d9b42ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:39.539347 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a70126-0f05-4f30-94a8-67745d9b42ee admin admin] PATH_INFO: `/v3/projects/d1ed86c2d71446e5a6f01d2a3da59ab4/users/ea2c355148144395b7e1357089e770b8/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:39.556710 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a70126-0f05-4f30-94a8-67745d9b42ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1ed86c2d71446e5a6f01d2a3da59ab4, user_id=ea2c355148144395b7e1357089e770b8, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:39.559622 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a70126-0f05-4f30-94a8-67745d9b42ee admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:39.574099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1061/2098] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:39 2022] PUT /identity/v3/projects/d1ed86c2d71446e5a6f01d2a3da59ab4/users/ea2c355148144395b7e1357089e770b8/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:39.578948 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ea324-85ad-44c3-a9b0-53230e3ceada admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:39.584484 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-614ea324-85ad-44c3-a9b0-53230e3ceada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:39.585253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8b86a-09f7-4655-a624-09e239726429 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:39.585798 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8b86a-09f7-4655-a624-09e239726429 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:39.587010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b8b86a-09f7-4655-a624-09e239726429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:39.600071 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ea324-85ad-44c3-a9b0-53230e3ceada admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:39.618755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-35b8b86a-09f7-4655-a624-09e239726429 None None] MFA Rules not processed for user `ea2c355148144395b7e1357089e770b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:39.619828 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-35b8b86a-09f7-4655-a624-09e239726429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:39.676316 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc869944-77b7-49d3-9cd6-d290ccc92e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:39.677226 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc869944-77b7-49d3-9cd6-d290ccc92e9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:39.677515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc869944-77b7-49d3-9cd6-d290ccc92e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:39.678199 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc869944-77b7-49d3-9cd6-d290ccc92e9b admin admin] PATH_INFO: `/v3/users/5e998d8a00e942818aaa93296f828219` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:39.679860 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc869944-77b7-49d3-9cd6-d290ccc92e9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e998d8a00e942818aaa93296f828219)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:39.682629 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc869944-77b7-49d3-9cd6-d290ccc92e9b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:39.711288 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1062/2099] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:39.727086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8b86a-09f7-4655-a624-09e239726429 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:39.728910 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-35b8b86a-09f7-4655-a624-09e239726429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:39.752675 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-fc869944-77b7-49d3-9cd6-d290ccc92e9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e998d8a00e942818aaa93296f828219'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:39.753516 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-fc869944-77b7-49d3-9cd6-d290ccc92e9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e998d8a00e942818aaa93296f828219'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:39.762821 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-fc869944-77b7-49d3-9cd6-d290ccc92e9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e998d8a00e942818aaa93296f828219'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:39.763827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-fc869944-77b7-49d3-9cd6-d290ccc92e9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e998d8a00e942818aaa93296f828219'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:39.778180 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-fc869944-77b7-49d3-9cd6-d290ccc92e9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e998d8a00e942818aaa93296f828219'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:39.796581 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1038/2100] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:39 2022] DELETE /identity/v3/users/5e998d8a00e942818aaa93296f828219 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:39.800915 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d2acb-6445-4d61-9d67-89f9fae77148 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:39.801260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d2acb-6445-4d61-9d67-89f9fae77148 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:39.801459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b8b86a-09f7-4655-a624-09e239726429 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:39.801663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5d2acb-6445-4d61-9d67-89f9fae77148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:39.808363 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:39.822748 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-fe5d2acb-6445-4d61-9d67-89f9fae77148 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:39.824512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-fe5d2acb-6445-4d61-9d67-89f9fae77148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:39.870256 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924af30-a47c-4a09-a13e-1335f28af695 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:39.870896 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924af30-a47c-4a09-a13e-1335f28af695 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:39.871309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924af30-a47c-4a09-a13e-1335f28af695 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:39.871597 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7924af30-a47c-4a09-a13e-1335f28af695 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:39.873458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7924af30-a47c-4a09-a13e-1335f28af695 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:39.893996 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1039/2101] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:39.895618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:39.903737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d2acb-6445-4d61-9d67-89f9fae77148 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:39.905500 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-fe5d2acb-6445-4d61-9d67-89f9fae77148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:39.937400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924af30-a47c-4a09-a13e-1335f28af695 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:39.938849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7924af30-a47c-4a09-a13e-1335f28af695 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:39.948085 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1063/2102] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:02:39.963080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5d2acb-6445-4d61-9d67-89f9fae77148 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:39.966808 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69921d5c-fd67-418a-9c38-e063e9d78544 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:39.967492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69921d5c-fd67-418a-9c38-e063e9d78544 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:39.967807 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69921d5c-fd67-418a-9c38-e063e9d78544 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:39.968094 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69921d5c-fd67-418a-9c38-e063e9d78544 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:39.978433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69921d5c-fd67-418a-9c38-e063e9d78544 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:39.979910 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69921d5c-fd67-418a-9c38-e063e9d78544 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:39.989740 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1040/2103] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:02:40.288052 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924af30-a47c-4a09-a13e-1335f28af695 service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:40.289599 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7924af30-a47c-4a09-a13e-1335f28af695 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:40.305764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7924af30-a47c-4a09-a13e-1335f28af695 service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:40.310928 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf669b-313d-4b98-a4ed-6fc7bff2502c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:40.311836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf669b-313d-4b98-a4ed-6fc7bff2502c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:40.312127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf669b-313d-4b98-a4ed-6fc7bff2502c admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:40.312410 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cf669b-313d-4b98-a4ed-6fc7bff2502c admin admin] PATH_INFO: `/v3/projects/a277d0f4925c4e43bf58cb7d8e0465e9` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:40.313608 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf669b-313d-4b98-a4ed-6fc7bff2502c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a277d0f4925c4e43bf58cb7d8e0465e9)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:40.316192 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cf669b-313d-4b98-a4ed-6fc7bff2502c admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:40.366447 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a3cf669b-313d-4b98-a4ed-6fc7bff2502c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a277d0f4925c4e43bf58cb7d8e0465e9', 'request_id': 'req-a3cf669b-313d-4b98-a4ed-6fc7bff2502c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:40.372552 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a3cf669b-313d-4b98-a4ed-6fc7bff2502c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a277d0f4925c4e43bf58cb7d8e0465e9', 'request_id': 'req-a3cf669b-313d-4b98-a4ed-6fc7bff2502c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:40.378226 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a3cf669b-313d-4b98-a4ed-6fc7bff2502c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a277d0f4925c4e43bf58cb7d8e0465e9', 'request_id': 'req-a3cf669b-313d-4b98-a4ed-6fc7bff2502c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:40.391210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a3cf669b-313d-4b98-a4ed-6fc7bff2502c admin admin] The token cache is being invalidate because project a277d0f4925c4e43bf58cb7d8e0465e9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:40.391434 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a3cf669b-313d-4b98-a4ed-6fc7bff2502c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:40.392277 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a3cf669b-313d-4b98-a4ed-6fc7bff2502c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:40.393611 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1064/2104] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:40 2022] DELETE /identity/v3/projects/a277d0f4925c4e43bf58cb7d8e0465e9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:02:40.397573 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69921d5c-fd67-418a-9c38-e063e9d78544 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:40.408975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69921d5c-fd67-418a-9c38-e063e9d78544 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:40.465407 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cab468c-4182-4745-a784-e7a58efcb058 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:40.466283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab468c-4182-4745-a784-e7a58efcb058 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:40.466565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab468c-4182-4745-a784-e7a58efcb058 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:40.466898 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cab468c-4182-4745-a784-e7a58efcb058 admin admin] PATH_INFO: `/v3/users/7a68902bb9f64c998903fe36cbc7c607` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:40.468134 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab468c-4182-4745-a784-e7a58efcb058 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a68902bb9f64c998903fe36cbc7c607)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:40.470045 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cab468c-4182-4745-a784-e7a58efcb058 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:40.517562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5cab468c-4182-4745-a784-e7a58efcb058 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a68902bb9f64c998903fe36cbc7c607'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:40.518249 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5cab468c-4182-4745-a784-e7a58efcb058 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a68902bb9f64c998903fe36cbc7c607'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:40.519424 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:40.526571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5cab468c-4182-4745-a784-e7a58efcb058 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a68902bb9f64c998903fe36cbc7c607'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:40.527544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5cab468c-4182-4745-a784-e7a58efcb058 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a68902bb9f64c998903fe36cbc7c607'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:40.540400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5cab468c-4182-4745-a784-e7a58efcb058 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a68902bb9f64c998903fe36cbc7c607'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:40.554097 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1041/2105] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:40 2022] DELETE /identity/v3/users/7a68902bb9f64c998903fe36cbc7c607 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:40.825850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf669b-313d-4b98-a4ed-6fc7bff2502c admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:40.827323 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a3cf669b-313d-4b98-a4ed-6fc7bff2502c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:40.840573 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cf669b-313d-4b98-a4ed-6fc7bff2502c admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:40.902679 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e17bdda-4504-4ec6-9039-0b4f9e949416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:40.903507 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17bdda-4504-4ec6-9039-0b4f9e949416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:40.903796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17bdda-4504-4ec6-9039-0b4f9e949416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:40.904076 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e17bdda-4504-4ec6-9039-0b4f9e949416 admin admin] PATH_INFO: `/v3/projects/25b953f244224f3dbbe2e93caa096bf6` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:40.905275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e17bdda-4504-4ec6-9039-0b4f9e949416 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25b953f244224f3dbbe2e93caa096bf6)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:40.907274 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e17bdda-4504-4ec6-9039-0b4f9e949416 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:40.954891 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:40.956289 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-4e17bdda-4504-4ec6-9039-0b4f9e949416 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25b953f244224f3dbbe2e93caa096bf6', 'request_id': 'req-4e17bdda-4504-4ec6-9039-0b4f9e949416'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:40.964519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-4e17bdda-4504-4ec6-9039-0b4f9e949416 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25b953f244224f3dbbe2e93caa096bf6', 'request_id': 'req-4e17bdda-4504-4ec6-9039-0b4f9e949416'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:40.970889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-4e17bdda-4504-4ec6-9039-0b4f9e949416 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25b953f244224f3dbbe2e93caa096bf6', 'request_id': 'req-4e17bdda-4504-4ec6-9039-0b4f9e949416'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:40.985207 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-4e17bdda-4504-4ec6-9039-0b4f9e949416 admin admin] The token cache is being invalidate because project 25b953f244224f3dbbe2e93caa096bf6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:40.985501 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-4e17bdda-4504-4ec6-9039-0b4f9e949416 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:40.986174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-4e17bdda-4504-4ec6-9039-0b4f9e949416 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:40.987757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1065/2106] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:40 2022] DELETE /identity/v3/projects/25b953f244224f3dbbe2e93caa096bf6 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:40.993257 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6276dc-267a-4726-9d9d-d433b670d9c7 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:40.993546 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6276dc-267a-4726-9d9d-d433b670d9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:40.993823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6276dc-267a-4726-9d9d-d433b670d9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:41.011904 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-fc6276dc-267a-4726-9d9d-d433b670d9c7 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:41.013117 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-fc6276dc-267a-4726-9d9d-d433b670d9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:41.074166 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1042/2107] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:41.082080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02229f3c-d8c1-4a07-b3d3-1e61c2e22468 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:41.082366 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02229f3c-d8c1-4a07-b3d3-1e61c2e22468 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:41.082678 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02229f3c-d8c1-4a07-b3d3-1e61c2e22468 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:41.100535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-02229f3c-d8c1-4a07-b3d3-1e61c2e22468 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:41.101774 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-02229f3c-d8c1-4a07-b3d3-1e61c2e22468 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:41.109606 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1066/2108] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:41.113641 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6276dc-267a-4726-9d9d-d433b670d9c7 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:41.124654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6276dc-267a-4726-9d9d-d433b670d9c7 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:41.129324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0ee5b-029a-493e-ab51-42011bc88398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:41.129970 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0ee5b-029a-493e-ab51-42011bc88398 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:41.130255 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0ee5b-029a-493e-ab51-42011bc88398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:41.130681 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0ee5b-029a-493e-ab51-42011bc88398 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:41.131402 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0ee5b-029a-493e-ab51-42011bc88398 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:02:41.131713 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0ee5b-029a-493e-ab51-42011bc88398 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:41.133923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0ee5b-029a-493e-ab51-42011bc88398 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:41.145227 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1043/2109] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:02:41.149425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02229f3c-d8c1-4a07-b3d3-1e61c2e22468 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:41.158814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02229f3c-d8c1-4a07-b3d3-1e61c2e22468 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:41.163371 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7770e-d6ca-4f0a-b4d7-2a3957255e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:41.164013 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7770e-d6ca-4f0a-b4d7-2a3957255e4d admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:41.164830 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7770e-d6ca-4f0a-b4d7-2a3957255e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:41.165105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c7770e-d6ca-4f0a-b4d7-2a3957255e4d admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:41.165788 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7770e-d6ca-4f0a-b4d7-2a3957255e4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:41.167687 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c7770e-d6ca-4f0a-b4d7-2a3957255e4d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:41.194495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1067/2110] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:41 2022] POST /identity/v3/projects => generated 385 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:41.199456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0ee5b-029a-493e-ab51-42011bc88398 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:41.209908 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0ee5b-029a-493e-ab51-42011bc88398 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:41.267190 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97456d86-64c3-461e-a0e6-718d4fac03ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:41.267874 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97456d86-64c3-461e-a0e6-718d4fac03ee admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:41.268163 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97456d86-64c3-461e-a0e6-718d4fac03ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:41.268446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97456d86-64c3-461e-a0e6-718d4fac03ee admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:41.269128 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97456d86-64c3-461e-a0e6-718d4fac03ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:41.271096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97456d86-64c3-461e-a0e6-718d4fac03ee admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:41.298574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1044/2111] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:41 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:41.299807 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7770e-d6ca-4f0a-b4d7-2a3957255e4d admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:41.310780 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c7770e-d6ca-4f0a-b4d7-2a3957255e4d admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:41.315571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e459151-8b5e-40fa-b709-b0e845abf925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:41.316238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e459151-8b5e-40fa-b709-b0e845abf925 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:41.316523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e459151-8b5e-40fa-b709-b0e845abf925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:41.316801 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e459151-8b5e-40fa-b709-b0e845abf925 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:41.317526 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e459151-8b5e-40fa-b709-b0e845abf925 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:41.319144 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e459151-8b5e-40fa-b709-b0e845abf925 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:41.329501 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1068/2112] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:41.333499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97456d86-64c3-461e-a0e6-718d4fac03ee admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:41.343366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97456d86-64c3-461e-a0e6-718d4fac03ee admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:41.348034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913075ab-d569-4a50-9fda-d69f8e63b69c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:41.348691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913075ab-d569-4a50-9fda-d69f8e63b69c admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:41.348983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913075ab-d569-4a50-9fda-d69f8e63b69c admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:41.349257 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913075ab-d569-4a50-9fda-d69f8e63b69c admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:41.350233 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913075ab-d569-4a50-9fda-d69f8e63b69c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:41.352370 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913075ab-d569-4a50-9fda-d69f8e63b69c admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:41.362426 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1045/2113] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:41.366498 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e459151-8b5e-40fa-b709-b0e845abf925 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:41.376854 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e459151-8b5e-40fa-b709-b0e845abf925 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:41.381485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768b1b0-180a-4554-9770-49e350482f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:41.382356 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768b1b0-180a-4554-9770-49e350482f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:41.382683 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768b1b0-180a-4554-9770-49e350482f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:41.382992 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e768b1b0-180a-4554-9770-49e350482f6d admin admin] PATH_INFO: `/v3/projects/51825e12e6bc4cd3a1e71f2c8ecf9036/users/b8612f55581745f6ab7df3f0f5293373/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:41.398015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768b1b0-180a-4554-9770-49e350482f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=51825e12e6bc4cd3a1e71f2c8ecf9036, user_id=b8612f55581745f6ab7df3f0f5293373, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:41.400722 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e768b1b0-180a-4554-9770-49e350482f6d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:41.413307 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1069/2114] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:41 2022] PUT /identity/v3/projects/51825e12e6bc4cd3a1e71f2c8ecf9036/users/b8612f55581745f6ab7df3f0f5293373/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:41.425127 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e4a2a-b16e-4bf0-b8f9-c541e2bc86fb None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:41.425604 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e4a2a-b16e-4bf0-b8f9-c541e2bc86fb None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:41.425692 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914e4a2a-b16e-4bf0-b8f9-c541e2bc86fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:41.452288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-914e4a2a-b16e-4bf0-b8f9-c541e2bc86fb None None] MFA Rules not processed for user `b8612f55581745f6ab7df3f0f5293373`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:41.453489 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-914e4a2a-b16e-4bf0-b8f9-c541e2bc86fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:41.471799 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:41.522650 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1046/2115] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:41.527958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768b1b0-180a-4554-9770-49e350482f6d admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:41.589992 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e768b1b0-180a-4554-9770-49e350482f6d admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:41.594717 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f228ced-f164-4d57-8338-5d0ed0137404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:41.595415 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f228ced-f164-4d57-8338-5d0ed0137404 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:41.595730 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f228ced-f164-4d57-8338-5d0ed0137404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:41.596011 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f228ced-f164-4d57-8338-5d0ed0137404 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:41.596709 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f228ced-f164-4d57-8338-5d0ed0137404 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:41.598621 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f228ced-f164-4d57-8338-5d0ed0137404 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:41.627849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1070/2116] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:41 2022] POST /identity/v3/projects => generated 385 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:41.629640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e4a2a-b16e-4bf0-b8f9-c541e2bc86fb None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:41.641872 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914e4a2a-b16e-4bf0-b8f9-c541e2bc86fb None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:41.697734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e4f7c-ac46-4898-8a89-ba30299fa656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:41.698323 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e4f7c-ac46-4898-8a89-ba30299fa656 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:41.698644 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e4f7c-ac46-4898-8a89-ba30299fa656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:41.698917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37e4f7c-ac46-4898-8a89-ba30299fa656 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:41.699604 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e4f7c-ac46-4898-8a89-ba30299fa656 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:41.701510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37e4f7c-ac46-4898-8a89-ba30299fa656 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:41.721977 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1047/2117] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:41 2022] POST /identity/v3/users => generated 299 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:41.726086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f228ced-f164-4d57-8338-5d0ed0137404 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:41.736168 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f228ced-f164-4d57-8338-5d0ed0137404 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:41.740832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9f95a-8f55-4c93-aa99-14a60d792ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:41.741487 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9f95a-8f55-4c93-aa99-14a60d792ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:41.741773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9f95a-8f55-4c93-aa99-14a60d792ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:41.742047 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f9f95a-8f55-4c93-aa99-14a60d792ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:41.743327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9f95a-8f55-4c93-aa99-14a60d792ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:41.744901 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f9f95a-8f55-4c93-aa99-14a60d792ab2 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:41.755330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1071/2118] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:41.759800 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e4f7c-ac46-4898-8a89-ba30299fa656 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:41.769858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37e4f7c-ac46-4898-8a89-ba30299fa656 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:41.774691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6259f25-40cc-4847-b334-ae5c541f3905 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:41.775576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6259f25-40cc-4847-b334-ae5c541f3905 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:41.775861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6259f25-40cc-4847-b334-ae5c541f3905 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:41.776192 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6259f25-40cc-4847-b334-ae5c541f3905 admin admin] PATH_INFO: `/v3/projects/20e71a6807874d42a3f6905e4e564f2f/users/3f54bcfdc77e447492a19d6feb536819/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:41.791715 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6259f25-40cc-4847-b334-ae5c541f3905 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20e71a6807874d42a3f6905e4e564f2f, user_id=3f54bcfdc77e447492a19d6feb536819, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:41.794447 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6259f25-40cc-4847-b334-ae5c541f3905 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:41.807759 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1048/2119] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:41 2022] PUT /identity/v3/projects/20e71a6807874d42a3f6905e4e564f2f/users/3f54bcfdc77e447492a19d6feb536819/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:41.811605 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9f95a-8f55-4c93-aa99-14a60d792ab2 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:41.821585 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f9f95a-8f55-4c93-aa99-14a60d792ab2 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:41.865096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:41.878912 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f849bd4a-1715-426b-9e0d-13a88dbb50be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:41.879585 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849bd4a-1715-426b-9e0d-13a88dbb50be admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:41.879870 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849bd4a-1715-426b-9e0d-13a88dbb50be admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:41.880143 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f849bd4a-1715-426b-9e0d-13a88dbb50be admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:41.881002 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849bd4a-1715-426b-9e0d-13a88dbb50be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:41.882427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f849bd4a-1715-426b-9e0d-13a88dbb50be admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:41.893386 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1072/2120] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:41 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:41.897435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6259f25-40cc-4847-b334-ae5c541f3905 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:41.907786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6259f25-40cc-4847-b334-ae5c541f3905 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:41.912557 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d7181-391c-409e-b81e-c819ddd95d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:41.913435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d7181-391c-409e-b81e-c819ddd95d82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:41.913719 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d7181-391c-409e-b81e-c819ddd95d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:41.914001 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43d7181-391c-409e-b81e-c819ddd95d82 admin admin] PATH_INFO: `/v3/projects/20e71a6807874d42a3f6905e4e564f2f/users/3f54bcfdc77e447492a19d6feb536819/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:41.916499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d7181-391c-409e-b81e-c819ddd95d82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20e71a6807874d42a3f6905e4e564f2f, user_id=3f54bcfdc77e447492a19d6feb536819, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:41.919155 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43d7181-391c-409e-b81e-c819ddd95d82 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:41.929049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1049/2121] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:41 2022] PUT /identity/v3/projects/20e71a6807874d42a3f6905e4e564f2f/users/3f54bcfdc77e447492a19d6feb536819/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:41.939154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef1e72-cbfb-4335-a596-1d7829c762e6 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:41.939442 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef1e72-cbfb-4335-a596-1d7829c762e6 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:41.939726 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ef1e72-cbfb-4335-a596-1d7829c762e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:41.968391 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-68ef1e72-cbfb-4335-a596-1d7829c762e6 None None] MFA Rules not processed for user `3f54bcfdc77e447492a19d6feb536819`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:41.969599 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-68ef1e72-cbfb-4335-a596-1d7829c762e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:42.039710 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1073/2122] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:42.050974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d7181-391c-409e-b81e-c819ddd95d82 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:42.052995 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d43d7181-391c-409e-b81e-c819ddd95d82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:42.124927 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43d7181-391c-409e-b81e-c819ddd95d82 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:42.128868 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ca2d6c-82bf-4e4a-bb5c-5a00cd2c427f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:42.129524 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca2d6c-82bf-4e4a-bb5c-5a00cd2c427f service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:42.130001 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca2d6c-82bf-4e4a-bb5c-5a00cd2c427f service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:42.130108 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ca2d6c-82bf-4e4a-bb5c-5a00cd2c427f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:42.162253 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:42.204045 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca2d6c-82bf-4e4a-bb5c-5a00cd2c427f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:42.215070 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ca2d6c-82bf-4e4a-bb5c-5a00cd2c427f service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:42.236897 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1050/2123] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:42 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:47.459751 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef1e72-cbfb-4335-a596-1d7829c762e6 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:47.463150 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-68ef1e72-cbfb-4335-a596-1d7829c762e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:47.557115 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ef1e72-cbfb-4335-a596-1d7829c762e6 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:47.572332 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771434b-758f-4a26-ab3d-dcb07196e1be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:47.574162 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771434b-758f-4a26-ab3d-dcb07196e1be admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:47.574510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771434b-758f-4a26-ab3d-dcb07196e1be admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:47.574919 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1771434b-758f-4a26-ab3d-dcb07196e1be admin admin] PATH_INFO: `/v3/users/cca3974ca425405caa7bd3dd7b5b3555` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:47.576240 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771434b-758f-4a26-ab3d-dcb07196e1be admin admin] RBAC: Authorizing `identity:delete_user(user_id=cca3974ca425405caa7bd3dd7b5b3555)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:47.578360 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1771434b-758f-4a26-ab3d-dcb07196e1be admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:47.642866 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1771434b-758f-4a26-ab3d-dcb07196e1be admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cca3974ca425405caa7bd3dd7b5b3555'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:47.652270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1771434b-758f-4a26-ab3d-dcb07196e1be admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cca3974ca425405caa7bd3dd7b5b3555'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:47.652994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1771434b-758f-4a26-ab3d-dcb07196e1be admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cca3974ca425405caa7bd3dd7b5b3555'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:47.665710 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:47.666990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1771434b-758f-4a26-ab3d-dcb07196e1be admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cca3974ca425405caa7bd3dd7b5b3555'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:47.667993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1771434b-758f-4a26-ab3d-dcb07196e1be admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cca3974ca425405caa7bd3dd7b5b3555'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:47.684816 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1074/2124] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:47 2022] DELETE /identity/v3/users/cca3974ca425405caa7bd3dd7b5b3555 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:02:47.689235 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2093a-340a-49be-bea0-3eadb0ac8dc1 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:47.689752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2093a-340a-49be-bea0-3eadb0ac8dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:47.689947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf2093a-340a-49be-bea0-3eadb0ac8dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:47.709074 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-caf2093a-340a-49be-bea0-3eadb0ac8dc1 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:47.710353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-caf2093a-340a-49be-bea0-3eadb0ac8dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:47.811661 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1051/2125] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:47.822423 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771434b-758f-4a26-ab3d-dcb07196e1be admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:47.824298 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-1771434b-758f-4a26-ab3d-dcb07196e1be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:47.900256 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1771434b-758f-4a26-ab3d-dcb07196e1be admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:47.904539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58764202-60f8-4e4e-aacf-3198ce6ee626 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:47.905233 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58764202-60f8-4e4e-aacf-3198ce6ee626 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:47.905764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58764202-60f8-4e4e-aacf-3198ce6ee626 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:47.906095 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58764202-60f8-4e4e-aacf-3198ce6ee626 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:47.918537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58764202-60f8-4e4e-aacf-3198ce6ee626 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:47.920109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58764202-60f8-4e4e-aacf-3198ce6ee626 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:47.931769 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1075/2126] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:48.316426 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2093a-340a-49be-bea0-3eadb0ac8dc1 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:48.317970 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-caf2093a-340a-49be-bea0-3eadb0ac8dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:48.337385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf2093a-340a-49be-bea0-3eadb0ac8dc1 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:48.358279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140d1d8-ff0f-437e-b9d3-23d904742cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:48.359461 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140d1d8-ff0f-437e-b9d3-23d904742cff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:48.359821 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140d1d8-ff0f-437e-b9d3-23d904742cff admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:48.360208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6140d1d8-ff0f-437e-b9d3-23d904742cff admin admin] PATH_INFO: `/v3/projects/71182a78451c403fb266ac5577f00a40` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:48.361617 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140d1d8-ff0f-437e-b9d3-23d904742cff admin admin] RBAC: Authorizing `identity:delete_project(project_id=71182a78451c403fb266ac5577f00a40)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:48.365905 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6140d1d8-ff0f-437e-b9d3-23d904742cff admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:48.429395 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6140d1d8-ff0f-437e-b9d3-23d904742cff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71182a78451c403fb266ac5577f00a40', 'request_id': 'req-6140d1d8-ff0f-437e-b9d3-23d904742cff'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:48.436439 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6140d1d8-ff0f-437e-b9d3-23d904742cff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71182a78451c403fb266ac5577f00a40', 'request_id': 'req-6140d1d8-ff0f-437e-b9d3-23d904742cff'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:48.443338 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6140d1d8-ff0f-437e-b9d3-23d904742cff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71182a78451c403fb266ac5577f00a40', 'request_id': 'req-6140d1d8-ff0f-437e-b9d3-23d904742cff'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:48.458288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6140d1d8-ff0f-437e-b9d3-23d904742cff admin admin] The token cache is being invalidate because project 71182a78451c403fb266ac5577f00a40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:48.458958 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6140d1d8-ff0f-437e-b9d3-23d904742cff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:48.459980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6140d1d8-ff0f-437e-b9d3-23d904742cff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:48.463259 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1052/2127] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:48 2022] DELETE /identity/v3/projects/71182a78451c403fb266ac5577f00a40 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:48.688805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58764202-60f8-4e4e-aacf-3198ce6ee626 service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:48.690319 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-58764202-60f8-4e4e-aacf-3198ce6ee626 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:48.752053 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58764202-60f8-4e4e-aacf-3198ce6ee626 service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:48.807294 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cefc09-2c53-498a-ba55-a748f772e1f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:48.807956 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cefc09-2c53-498a-ba55-a748f772e1f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:48.808282 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cefc09-2c53-498a-ba55-a748f772e1f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:48.808599 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cefc09-2c53-498a-ba55-a748f772e1f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:48.810523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-13cefc09-2c53-498a-ba55-a748f772e1f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:48.874183 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:DELETE=2,keystone:SELECT=97,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:48.876727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cefc09-2c53-498a-ba55-a748f772e1f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:48.878699 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cefc09-2c53-498a-ba55-a748f772e1f4 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:48.890766 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1076/2128] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:48 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:49.003525 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140d1d8-ff0f-437e-b9d3-23d904742cff admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:49.005019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6140d1d8-ff0f-437e-b9d3-23d904742cff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:49.074673 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6140d1d8-ff0f-437e-b9d3-23d904742cff admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:49.079222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74847dee-21f3-4cea-abbd-b2c1fe0cc68d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:49.084605 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74847dee-21f3-4cea-abbd-b2c1fe0cc68d service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:49.084605 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74847dee-21f3-4cea-abbd-b2c1fe0cc68d service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:49.084605 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74847dee-21f3-4cea-abbd-b2c1fe0cc68d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:49.086467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-74847dee-21f3-4cea-abbd-b2c1fe0cc68d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:49.091823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:49.162866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74847dee-21f3-4cea-abbd-b2c1fe0cc68d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:49.164908 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74847dee-21f3-4cea-abbd-b2c1fe0cc68d service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:49.188547 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1053/2129] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:50.643419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cefc09-2c53-498a-ba55-a748f772e1f4 service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:50.645319 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-13cefc09-2c53-498a-ba55-a748f772e1f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:50.757150 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cefc09-2c53-498a-ba55-a748f772e1f4 service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:50.764076 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57753db0-4249-45c8-866a-dbf38ca135f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:50.765163 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57753db0-4249-45c8-866a-dbf38ca135f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:50.765342 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57753db0-4249-45c8-866a-dbf38ca135f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:50.765702 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57753db0-4249-45c8-866a-dbf38ca135f7 admin admin] PATH_INFO: `/v3/users/ea2c355148144395b7e1357089e770b8` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:50.767693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57753db0-4249-45c8-866a-dbf38ca135f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea2c355148144395b7e1357089e770b8)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:50.769471 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57753db0-4249-45c8-866a-dbf38ca135f7 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:50.857638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-57753db0-4249-45c8-866a-dbf38ca135f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea2c355148144395b7e1357089e770b8'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:50.877647 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-57753db0-4249-45c8-866a-dbf38ca135f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea2c355148144395b7e1357089e770b8'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:50.878673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-57753db0-4249-45c8-866a-dbf38ca135f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea2c355148144395b7e1357089e770b8'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:50.905425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-57753db0-4249-45c8-866a-dbf38ca135f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea2c355148144395b7e1357089e770b8'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:50.906300 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-57753db0-4249-45c8-866a-dbf38ca135f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea2c355148144395b7e1357089e770b8'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:50.924347 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1077/2130] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:50 2022] DELETE /identity/v3/users/ea2c355148144395b7e1357089e770b8 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:50.929312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bee5e6-33ed-4232-a28e-9ccff01e83bc None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:50.929855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bee5e6-33ed-4232-a28e-9ccff01e83bc None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:50.930292 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bee5e6-33ed-4232-a28e-9ccff01e83bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:50.959442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-43bee5e6-33ed-4232-a28e-9ccff01e83bc None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:50.961125 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-43bee5e6-33ed-4232-a28e-9ccff01e83bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:51.033676 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1054/2131] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:51.044058 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57753db0-4249-45c8-866a-dbf38ca135f7 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:51.045531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-57753db0-4249-45c8-866a-dbf38ca135f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:51.105615 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57753db0-4249-45c8-866a-dbf38ca135f7 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:51.109481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b14bed7-4dff-434a-86a7-4caa01fc71d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:51.110257 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14bed7-4dff-434a-86a7-4caa01fc71d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:51.110646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14bed7-4dff-434a-86a7-4caa01fc71d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:51.111002 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b14bed7-4dff-434a-86a7-4caa01fc71d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:51.120722 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14bed7-4dff-434a-86a7-4caa01fc71d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:51.122268 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b14bed7-4dff-434a-86a7-4caa01fc71d3 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:51.140764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1078/2132] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:02:51.740374 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bee5e6-33ed-4232-a28e-9ccff01e83bc None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:51.742340 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-43bee5e6-33ed-4232-a28e-9ccff01e83bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:51.772313 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bee5e6-33ed-4232-a28e-9ccff01e83bc None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:51.778137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ac82f-0fdc-48f0-b2c5-42bba6a6c572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:51.779065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ac82f-0fdc-48f0-b2c5-42bba6a6c572 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:51.780046 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ac82f-0fdc-48f0-b2c5-42bba6a6c572 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:51.780334 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ac82f-0fdc-48f0-b2c5-42bba6a6c572 admin admin] PATH_INFO: `/v3/projects/d1ed86c2d71446e5a6f01d2a3da59ab4` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:51.781639 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ac82f-0fdc-48f0-b2c5-42bba6a6c572 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1ed86c2d71446e5a6f01d2a3da59ab4)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:51.783530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ac82f-0fdc-48f0-b2c5-42bba6a6c572 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:51.862418 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6c1ac82f-0fdc-48f0-b2c5-42bba6a6c572 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1ed86c2d71446e5a6f01d2a3da59ab4', 'request_id': 'req-6c1ac82f-0fdc-48f0-b2c5-42bba6a6c572'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:51.871235 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6c1ac82f-0fdc-48f0-b2c5-42bba6a6c572 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1ed86c2d71446e5a6f01d2a3da59ab4', 'request_id': 'req-6c1ac82f-0fdc-48f0-b2c5-42bba6a6c572'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:51.887366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6c1ac82f-0fdc-48f0-b2c5-42bba6a6c572 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1ed86c2d71446e5a6f01d2a3da59ab4', 'request_id': 'req-6c1ac82f-0fdc-48f0-b2c5-42bba6a6c572'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:51.910841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6c1ac82f-0fdc-48f0-b2c5-42bba6a6c572 admin admin] The token cache is being invalidate because project d1ed86c2d71446e5a6f01d2a3da59ab4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:51.911352 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6c1ac82f-0fdc-48f0-b2c5-42bba6a6c572 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:51.924582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-6c1ac82f-0fdc-48f0-b2c5-42bba6a6c572 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:51.928176 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1055/2133] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:51 2022] DELETE /identity/v3/projects/d1ed86c2d71446e5a6f01d2a3da59ab4 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:51.933675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd06cbe-6f19-46dd-bfa3-87b93896ea0d None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:51.934688 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd06cbe-6f19-46dd-bfa3-87b93896ea0d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:51.935157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd06cbe-6f19-46dd-bfa3-87b93896ea0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:51.957844 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-0cd06cbe-6f19-46dd-bfa3-87b93896ea0d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:51.960579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-0cd06cbe-6f19-46dd-bfa3-87b93896ea0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:51.984511 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:52.063455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1079/2134] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:52.065037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0df3b3-a2cd-42e2-acde-92df68196c58 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:52.065502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0df3b3-a2cd-42e2-acde-92df68196c58 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:52.065608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0df3b3-a2cd-42e2-acde-92df68196c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:52.087465 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-cb0df3b3-a2cd-42e2-acde-92df68196c58 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:52.088938 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-cb0df3b3-a2cd-42e2-acde-92df68196c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:52.097394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1056/2135] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:52.102019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd06cbe-6f19-46dd-bfa3-87b93896ea0d None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:52.112547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd06cbe-6f19-46dd-bfa3-87b93896ea0d None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:52.119282 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab175d-717f-41c1-b2fb-640bc4390566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:52.119693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab175d-717f-41c1-b2fb-640bc4390566 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:52.119975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab175d-717f-41c1-b2fb-640bc4390566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:52.120261 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab175d-717f-41c1-b2fb-640bc4390566 admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:52.121107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab175d-717f-41c1-b2fb-640bc4390566 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:02:52.121429 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab175d-717f-41c1-b2fb-640bc4390566 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:52.123130 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab175d-717f-41c1-b2fb-640bc4390566 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:52.139111 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1080/2136] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:02:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 23:02:52.143645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0df3b3-a2cd-42e2-acde-92df68196c58 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:52.153574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0df3b3-a2cd-42e2-acde-92df68196c58 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:52.173136 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3620952-1af1-49fd-899a-a02578e234d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:52.174305 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3620952-1af1-49fd-899a-a02578e234d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:52.174960 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3620952-1af1-49fd-899a-a02578e234d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:52.175434 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3620952-1af1-49fd-899a-a02578e234d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:52.176397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3620952-1af1-49fd-899a-a02578e234d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:52.179115 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3620952-1af1-49fd-899a-a02578e234d6 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:52.197257 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:52.221060 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1057/2137] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:52 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:52.225627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab175d-717f-41c1-b2fb-640bc4390566 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:52.242975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab175d-717f-41c1-b2fb-640bc4390566 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:52.308393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd9517-3d50-488d-82b5-4e5271ba3777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:52.309238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd9517-3d50-488d-82b5-4e5271ba3777 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:52.309562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd9517-3d50-488d-82b5-4e5271ba3777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:52.309806 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bd9517-3d50-488d-82b5-4e5271ba3777 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:52.310517 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd9517-3d50-488d-82b5-4e5271ba3777 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:52.312518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bd9517-3d50-488d-82b5-4e5271ba3777 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:52.338510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1081/2138] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:52 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:02:52.339902 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3620952-1af1-49fd-899a-a02578e234d6 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:52.351991 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3620952-1af1-49fd-899a-a02578e234d6 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:52.357871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7da3f6-6a80-4722-a055-4623904e9444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:52.358559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7da3f6-6a80-4722-a055-4623904e9444 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:52.358878 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7da3f6-6a80-4722-a055-4623904e9444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:52.359154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7da3f6-6a80-4722-a055-4623904e9444 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:52.359897 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7da3f6-6a80-4722-a055-4623904e9444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:52.363219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7da3f6-6a80-4722-a055-4623904e9444 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:52.386556 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1058/2139] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:02:52.390212 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd9517-3d50-488d-82b5-4e5271ba3777 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:52.402113 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bd9517-3d50-488d-82b5-4e5271ba3777 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:52.407163 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1e456-b21f-4412-9cef-ddc460c596a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:52.407843 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1e456-b21f-4412-9cef-ddc460c596a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:52.408129 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1e456-b21f-4412-9cef-ddc460c596a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:52.408411 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1e456-b21f-4412-9cef-ddc460c596a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:52.409137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1e456-b21f-4412-9cef-ddc460c596a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:52.410764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1e456-b21f-4412-9cef-ddc460c596a5 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:52.422884 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1082/2140] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:52.426926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7da3f6-6a80-4722-a055-4623904e9444 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:52.437974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7da3f6-6a80-4722-a055-4623904e9444 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:52.442737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25649c-3e3d-4398-ba76-4def19bfe7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:52.443632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25649c-3e3d-4398-ba76-4def19bfe7de admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:52.443917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25649c-3e3d-4398-ba76-4def19bfe7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:52.444276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce25649c-3e3d-4398-ba76-4def19bfe7de admin admin] PATH_INFO: `/v3/projects/e4e7461392bf4748bb1633205e738245/users/3ef45548e1004a3bb218c6b9f0bd55d0/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:52.461317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25649c-3e3d-4398-ba76-4def19bfe7de admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4e7461392bf4748bb1633205e738245, user_id=3ef45548e1004a3bb218c6b9f0bd55d0, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:52.464063 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce25649c-3e3d-4398-ba76-4def19bfe7de admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:52.481416 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1059/2141] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:52 2022] PUT /identity/v3/projects/e4e7461392bf4748bb1633205e738245/users/3ef45548e1004a3bb218c6b9f0bd55d0/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:52.492483 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fba304-5c7e-48e4-8a84-5d888de78def None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:52.492777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fba304-5c7e-48e4-8a84-5d888de78def None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:52.494746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fba304-5c7e-48e4-8a84-5d888de78def None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:52.525202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-e2fba304-5c7e-48e4-8a84-5d888de78def None None] MFA Rules not processed for user `3ef45548e1004a3bb218c6b9f0bd55d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:52.526939 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e2fba304-5c7e-48e4-8a84-5d888de78def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:52.610706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1083/2142] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:52.618669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25649c-3e3d-4398-ba76-4def19bfe7de admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:52.620876 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-ce25649c-3e3d-4398-ba76-4def19bfe7de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:52.692147 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce25649c-3e3d-4398-ba76-4def19bfe7de admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:52.698041 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e83fe3-a0d1-4efa-82f9-9d67227356e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:52.698821 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e83fe3-a0d1-4efa-82f9-9d67227356e3 service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:52.699116 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e83fe3-a0d1-4efa-82f9-9d67227356e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:52.699429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e83fe3-a0d1-4efa-82f9-9d67227356e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:52.709105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e83fe3-a0d1-4efa-82f9-9d67227356e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:52.710648 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e83fe3-a0d1-4efa-82f9-9d67227356e3 service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:52.726858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1060/2143] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:53.970484 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fba304-5c7e-48e4-8a84-5d888de78def None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:53.973389 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e2fba304-5c7e-48e4-8a84-5d888de78def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:53.999379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:54.105302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fba304-5c7e-48e4-8a84-5d888de78def None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:54.111166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1121a-a4be-4215-8133-16e3a4aa5492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:54.112116 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1121a-a4be-4215-8133-16e3a4aa5492 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:54.112442 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1121a-a4be-4215-8133-16e3a4aa5492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:54.112721 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff1121a-a4be-4215-8133-16e3a4aa5492 admin admin] PATH_INFO: `/v3/users/b8612f55581745f6ab7df3f0f5293373` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:54.114217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1121a-a4be-4215-8133-16e3a4aa5492 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8612f55581745f6ab7df3f0f5293373)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:54.117496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff1121a-a4be-4215-8133-16e3a4aa5492 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:54.204224 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6ff1121a-a4be-4215-8133-16e3a4aa5492 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8612f55581745f6ab7df3f0f5293373'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:54.220383 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6ff1121a-a4be-4215-8133-16e3a4aa5492 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8612f55581745f6ab7df3f0f5293373'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:54.221382 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6ff1121a-a4be-4215-8133-16e3a4aa5492 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8612f55581745f6ab7df3f0f5293373'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:54.252441 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6ff1121a-a4be-4215-8133-16e3a4aa5492 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8612f55581745f6ab7df3f0f5293373'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:54.252797 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6ff1121a-a4be-4215-8133-16e3a4aa5492 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8612f55581745f6ab7df3f0f5293373'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:54.278522 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1084/2144] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:53 2022] DELETE /identity/v3/users/b8612f55581745f6ab7df3f0f5293373 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:54.282851 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13373ba8-e82b-4a48-a8aa-aac48766e648 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:54.283341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13373ba8-e82b-4a48-a8aa-aac48766e648 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:54.283618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13373ba8-e82b-4a48-a8aa-aac48766e648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:54.303953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-13373ba8-e82b-4a48-a8aa-aac48766e648 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:54.305362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-13373ba8-e82b-4a48-a8aa-aac48766e648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:54.490254 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1061/2145] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:54.504706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1121a-a4be-4215-8133-16e3a4aa5492 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:54.508155 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-6ff1121a-a4be-4215-8133-16e3a4aa5492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:54.634497 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff1121a-a4be-4215-8133-16e3a4aa5492 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:54.639004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529ddbd-4381-4cb4-bf08-e868f546b14d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:54.639780 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529ddbd-4381-4cb4-bf08-e868f546b14d service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:54.640179 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529ddbd-4381-4cb4-bf08-e868f546b14d service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:54.640560 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9529ddbd-4381-4cb4-bf08-e868f546b14d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:54.664296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529ddbd-4381-4cb4-bf08-e868f546b14d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:54.667571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9529ddbd-4381-4cb4-bf08-e868f546b14d service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:54.679333 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1085/2146] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:02:54.968655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13373ba8-e82b-4a48-a8aa-aac48766e648 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:54.970553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-13373ba8-e82b-4a48-a8aa-aac48766e648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:54.985886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13373ba8-e82b-4a48-a8aa-aac48766e648 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:54.990586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118d30e-fe0c-4ea1-816c-72d6409a13fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:54.991496 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118d30e-fe0c-4ea1-816c-72d6409a13fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:54.991781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118d30e-fe0c-4ea1-816c-72d6409a13fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:54.992060 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7118d30e-fe0c-4ea1-816c-72d6409a13fe admin admin] PATH_INFO: `/v3/projects/51825e12e6bc4cd3a1e71f2c8ecf9036` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:54.993761 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7118d30e-fe0c-4ea1-816c-72d6409a13fe admin admin] RBAC: Authorizing `identity:delete_project(project_id=51825e12e6bc4cd3a1e71f2c8ecf9036)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:54.995983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7118d30e-fe0c-4ea1-816c-72d6409a13fe admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:55.042755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-7118d30e-fe0c-4ea1-816c-72d6409a13fe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51825e12e6bc4cd3a1e71f2c8ecf9036', 'request_id': 'req-7118d30e-fe0c-4ea1-816c-72d6409a13fe'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.048751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-7118d30e-fe0c-4ea1-816c-72d6409a13fe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51825e12e6bc4cd3a1e71f2c8ecf9036', 'request_id': 'req-7118d30e-fe0c-4ea1-816c-72d6409a13fe'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.052971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:55.056045 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-7118d30e-fe0c-4ea1-816c-72d6409a13fe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51825e12e6bc4cd3a1e71f2c8ecf9036', 'request_id': 'req-7118d30e-fe0c-4ea1-816c-72d6409a13fe'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.068790 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-7118d30e-fe0c-4ea1-816c-72d6409a13fe admin admin] The token cache is being invalidate because project 51825e12e6bc4cd3a1e71f2c8ecf9036 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:55.069618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-7118d30e-fe0c-4ea1-816c-72d6409a13fe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.070237 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-7118d30e-fe0c-4ea1-816c-72d6409a13fe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.071796 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1062/2147] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:54 2022] DELETE /identity/v3/projects/51825e12e6bc4cd3a1e71f2c8ecf9036 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:55.075794 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529ddbd-4381-4cb4-bf08-e868f546b14d service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:55.085518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9529ddbd-4381-4cb4-bf08-e868f546b14d service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:55.117171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:55.143696 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6318e3-24de-4ab1-9404-1d84d059ccd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:55.144562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6318e3-24de-4ab1-9404-1d84d059ccd1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:55.144850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6318e3-24de-4ab1-9404-1d84d059ccd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:55.145128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6318e3-24de-4ab1-9404-1d84d059ccd1 admin admin] PATH_INFO: `/v3/users/3f54bcfdc77e447492a19d6feb536819` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:55.146306 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6318e3-24de-4ab1-9404-1d84d059ccd1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f54bcfdc77e447492a19d6feb536819)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:55.148293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6318e3-24de-4ab1-9404-1d84d059ccd1 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:55.196317 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-ef6318e3-24de-4ab1-9404-1d84d059ccd1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f54bcfdc77e447492a19d6feb536819'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.204020 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-ef6318e3-24de-4ab1-9404-1d84d059ccd1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f54bcfdc77e447492a19d6feb536819'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.204708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-ef6318e3-24de-4ab1-9404-1d84d059ccd1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f54bcfdc77e447492a19d6feb536819'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.225175 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-ef6318e3-24de-4ab1-9404-1d84d059ccd1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f54bcfdc77e447492a19d6feb536819'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.225906 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-ef6318e3-24de-4ab1-9404-1d84d059ccd1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f54bcfdc77e447492a19d6feb536819'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.240473 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1086/2148] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:55 2022] DELETE /identity/v3/users/3f54bcfdc77e447492a19d6feb536819 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:55.521449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118d30e-fe0c-4ea1-816c-72d6409a13fe admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:55.523036 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-7118d30e-fe0c-4ea1-816c-72d6409a13fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:55.536328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7118d30e-fe0c-4ea1-816c-72d6409a13fe admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:55.596060 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667a5e3-034a-462e-a49e-21c0134ef536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:55.597136 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667a5e3-034a-462e-a49e-21c0134ef536 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:55.597463 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667a5e3-034a-462e-a49e-21c0134ef536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:55.597838 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0667a5e3-034a-462e-a49e-21c0134ef536 admin admin] PATH_INFO: `/v3/projects/20e71a6807874d42a3f6905e4e564f2f` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:55.599457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667a5e3-034a-462e-a49e-21c0134ef536 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20e71a6807874d42a3f6905e4e564f2f)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:55.601896 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0667a5e3-034a-462e-a49e-21c0134ef536 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:55.654290 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0667a5e3-034a-462e-a49e-21c0134ef536 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20e71a6807874d42a3f6905e4e564f2f', 'request_id': 'req-0667a5e3-034a-462e-a49e-21c0134ef536'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.661055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0667a5e3-034a-462e-a49e-21c0134ef536 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20e71a6807874d42a3f6905e4e564f2f', 'request_id': 'req-0667a5e3-034a-462e-a49e-21c0134ef536'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.667717 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0667a5e3-034a-462e-a49e-21c0134ef536 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20e71a6807874d42a3f6905e4e564f2f', 'request_id': 'req-0667a5e3-034a-462e-a49e-21c0134ef536'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.683382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0667a5e3-034a-462e-a49e-21c0134ef536 admin admin] The token cache is being invalidate because project 20e71a6807874d42a3f6905e4e564f2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:55.683850 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0667a5e3-034a-462e-a49e-21c0134ef536 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.684532 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0667a5e3-034a-462e-a49e-21c0134ef536 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.686336 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1063/2149] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:55 2022] DELETE /identity/v3/projects/20e71a6807874d42a3f6905e4e564f2f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:55.692308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe79e01-3e72-4821-8c27-6f76d3c0401a None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:55.692604 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe79e01-3e72-4821-8c27-6f76d3c0401a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:55.692886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe79e01-3e72-4821-8c27-6f76d3c0401a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:55.704809 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667a5e3-034a-462e-a49e-21c0134ef536 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:55.706867 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-0667a5e3-034a-462e-a49e-21c0134ef536 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:55.711815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-fbe79e01-3e72-4821-8c27-6f76d3c0401a None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:55.713019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-fbe79e01-3e72-4821-8c27-6f76d3c0401a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:55.788299 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1087/2150] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:55.796507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0667a5e3-034a-462e-a49e-21c0134ef536 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:55.796753 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9e6d1-cb7b-40e5-bffb-4ad6be4f6084 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:55.797058 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9e6d1-cb7b-40e5-bffb-4ad6be4f6084 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:55.797345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f9e6d1-cb7b-40e5-bffb-4ad6be4f6084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:55.801888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f6353e-167b-4bce-a33f-7aef67f7c65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:55.802970 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6353e-167b-4bce-a33f-7aef67f7c65c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:55.804046 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6353e-167b-4bce-a33f-7aef67f7c65c admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:55.804477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f6353e-167b-4bce-a33f-7aef67f7c65c admin admin] PATH_INFO: `/v3/users/3ef45548e1004a3bb218c6b9f0bd55d0` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:55.805893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6353e-167b-4bce-a33f-7aef67f7c65c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ef45548e1004a3bb218c6b9f0bd55d0)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:55.807858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f6353e-167b-4bce-a33f-7aef67f7c65c admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:55.817568 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-e5f9e6d1-cb7b-40e5-bffb-4ad6be4f6084 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:55.818882 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e5f9e6d1-cb7b-40e5-bffb-4ad6be4f6084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:55.828342 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1088/2151] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:55.832259 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9e6d1-cb7b-40e5-bffb-4ad6be4f6084 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:55.836250 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:55.848895 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f9e6d1-cb7b-40e5-bffb-4ad6be4f6084 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:55.854423 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebe30f-9758-48cd-bdb3-39cf7fd7738a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:55.854985 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebe30f-9758-48cd-bdb3-39cf7fd7738a admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:55.855275 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebe30f-9758-48cd-bdb3-39cf7fd7738a admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:55.855555 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ebe30f-9758-48cd-bdb3-39cf7fd7738a admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:55.856283 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebe30f-9758-48cd-bdb3-39cf7fd7738a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:02:55.856598 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebe30f-9758-48cd-bdb3-39cf7fd7738a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:55.858206 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ebe30f-9758-48cd-bdb3-39cf7fd7738a admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:55.868744 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-99f6353e-167b-4bce-a33f-7aef67f7c65c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ef45548e1004a3bb218c6b9f0bd55d0'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.869448 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-99f6353e-167b-4bce-a33f-7aef67f7c65c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ef45548e1004a3bb218c6b9f0bd55d0'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.872755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1089/2152] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:02:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 23:02:55.877685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebe30f-9758-48cd-bdb3-39cf7fd7738a admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:55.879010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-99f6353e-167b-4bce-a33f-7aef67f7c65c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ef45548e1004a3bb218c6b9f0bd55d0'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.879298 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c6ebe30f-9758-48cd-bdb3-39cf7fd7738a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:55.879835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-99f6353e-167b-4bce-a33f-7aef67f7c65c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ef45548e1004a3bb218c6b9f0bd55d0'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.893315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-99f6353e-167b-4bce-a33f-7aef67f7c65c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ef45548e1004a3bb218c6b9f0bd55d0'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:55.894079 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ebe30f-9758-48cd-bdb3-39cf7fd7738a admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:55.900389 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb292c-a6be-4e9c-8495-5d524c4c6561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:55.901312 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb292c-a6be-4e9c-8495-5d524c4c6561 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:55.901593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb292c-a6be-4e9c-8495-5d524c4c6561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:55.901893 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebb292c-a6be-4e9c-8495-5d524c4c6561 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:55.902800 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb292c-a6be-4e9c-8495-5d524c4c6561 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:55.904906 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebb292c-a6be-4e9c-8495-5d524c4c6561 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:55.910409 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1064/2153] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:02:55 2022] DELETE /identity/v3/users/3ef45548e1004a3bb218c6b9f0bd55d0 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:02:55.914994 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdffbb7-963f-45ed-8ad0-c4b463d55b4e None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:55.915284 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdffbb7-963f-45ed-8ad0-c4b463d55b4e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:55.915563 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdffbb7-963f-45ed-8ad0-c4b463d55b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:55.932770 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1090/2154] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:55 2022] POST /identity/v3/projects => generated 383 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:02:55.932960 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-dcdffbb7-963f-45ed-8ad0-c4b463d55b4e None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:55.934133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-dcdffbb7-963f-45ed-8ad0-c4b463d55b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:55.936540 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb292c-a6be-4e9c-8495-5d524c4c6561 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:55.950457 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:55.997750 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1065/2155] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:56.006010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebb292c-a6be-4e9c-8495-5d524c4c6561 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:56.008194 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdffbb7-963f-45ed-8ad0-c4b463d55b4e None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:56.009560 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-dcdffbb7-963f-45ed-8ad0-c4b463d55b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:56.011048 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71e0af-dc32-4064-ad36-b187a949522c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:56.011735 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71e0af-dc32-4064-ad36-b187a949522c admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:56.012055 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71e0af-dc32-4064-ad36-b187a949522c admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:56.012341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc71e0af-dc32-4064-ad36-b187a949522c admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:56.013038 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc71e0af-dc32-4064-ad36-b187a949522c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:56.015095 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc71e0af-dc32-4064-ad36-b187a949522c admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:56.038949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1091/2156] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:55 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:02:56.039958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71e0af-dc32-4064-ad36-b187a949522c admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:56.049407 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc71e0af-dc32-4064-ad36-b187a949522c admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:56.054354 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0f4ae-029c-4682-b1b1-c2195171ccd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:56.055053 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0f4ae-029c-4682-b1b1-c2195171ccd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:56.055364 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0f4ae-029c-4682-b1b1-c2195171ccd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:56.055618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b0f4ae-029c-4682-b1b1-c2195171ccd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:56.056344 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b0f4ae-029c-4682-b1b1-c2195171ccd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:56.057936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b0f4ae-029c-4682-b1b1-c2195171ccd4 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:56.071993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1092/2157] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:02:56.072159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcdffbb7-963f-45ed-8ad0-c4b463d55b4e None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:56.073074 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0f4ae-029c-4682-b1b1-c2195171ccd4 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:56.074002 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1636f49-7a38-4d12-b623-c62c2f0681a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:56.074685 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1636f49-7a38-4d12-b623-c62c2f0681a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:56.075004 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1636f49-7a38-4d12-b623-c62c2f0681a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:56.075284 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1636f49-7a38-4d12-b623-c62c2f0681a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:56.083299 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b0f4ae-029c-4682-b1b1-c2195171ccd4 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:56.085055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1636f49-7a38-4d12-b623-c62c2f0681a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:56.086487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1636f49-7a38-4d12-b623-c62c2f0681a7 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:56.088567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcace8a-b93e-4d04-8fd9-f41b4216876d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:56.089216 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcace8a-b93e-4d04-8fd9-f41b4216876d admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:56.089540 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcace8a-b93e-4d04-8fd9-f41b4216876d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:56.089823 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcace8a-b93e-4d04-8fd9-f41b4216876d admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:56.090536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcace8a-b93e-4d04-8fd9-f41b4216876d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:56.092140 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcace8a-b93e-4d04-8fd9-f41b4216876d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:56.096034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1066/2158] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:02:56.103348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1093/2159] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:56.107161 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1636f49-7a38-4d12-b623-c62c2f0681a7 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:56.117384 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1636f49-7a38-4d12-b623-c62c2f0681a7 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:56.122013 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27488cdf-d5a9-4810-a855-7c171d7aeeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:56.122945 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27488cdf-d5a9-4810-a855-7c171d7aeeab admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:56.123238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27488cdf-d5a9-4810-a855-7c171d7aeeab admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:56.123662 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27488cdf-d5a9-4810-a855-7c171d7aeeab admin admin] PATH_INFO: `/v3/projects/9761759a69724a21b0b9315a931b106f/users/69163c28aa75492aa01c70532bed15a9/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:56.139321 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27488cdf-d5a9-4810-a855-7c171d7aeeab admin admin] RBAC: Authorizing `identity:create_grant(project_id=9761759a69724a21b0b9315a931b106f, user_id=69163c28aa75492aa01c70532bed15a9, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:56.142398 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27488cdf-d5a9-4810-a855-7c171d7aeeab admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:56.156265 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1067/2160] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:56 2022] PUT /identity/v3/projects/9761759a69724a21b0b9315a931b106f/users/69163c28aa75492aa01c70532bed15a9/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:56.161690 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf42506-c563-47d6-af54-a744177af0dd None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:56.162002 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf42506-c563-47d6-af54-a744177af0dd None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:56.162307 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf42506-c563-47d6-af54-a744177af0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:56.180442 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-7cf42506-c563-47d6-af54-a744177af0dd None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:56.181652 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7cf42506-c563-47d6-af54-a744177af0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:56.281589 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1094/2161] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:56.287516 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27488cdf-d5a9-4810-a855-7c171d7aeeab admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:56.289315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-27488cdf-d5a9-4810-a855-7c171d7aeeab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:56.308735 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:56.351217 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27488cdf-d5a9-4810-a855-7c171d7aeeab admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:56.354941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3e2a0-94e8-4d10-9ec9-06f6ccca0877 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:56.355717 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3e2a0-94e8-4d10-9ec9-06f6ccca0877 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:56.356055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3e2a0-94e8-4d10-9ec9-06f6ccca0877 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:56.356341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3e2a0-94e8-4d10-9ec9-06f6ccca0877 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:56.365625 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d3e2a0-94e8-4d10-9ec9-06f6ccca0877 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:56.367095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d3e2a0-94e8-4d10-9ec9-06f6ccca0877 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:56.376899 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1068/2162] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:02:56.404004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf42506-c563-47d6-af54-a744177af0dd None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:56.405551 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7cf42506-c563-47d6-af54-a744177af0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:56.419955 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf42506-c563-47d6-af54-a744177af0dd None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:56.425787 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca1b99-a2c4-4c9d-bf4d-b5a743ae3e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:56.426715 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca1b99-a2c4-4c9d-bf4d-b5a743ae3e89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:56.427023 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca1b99-a2c4-4c9d-bf4d-b5a743ae3e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:56.427306 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca1b99-a2c4-4c9d-bf4d-b5a743ae3e89 admin admin] PATH_INFO: `/v3/projects/e4e7461392bf4748bb1633205e738245` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:56.428644 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca1b99-a2c4-4c9d-bf4d-b5a743ae3e89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4e7461392bf4748bb1633205e738245)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:56.430613 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fca1b99-a2c4-4c9d-bf4d-b5a743ae3e89 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:56.486171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3fca1b99-a2c4-4c9d-bf4d-b5a743ae3e89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4e7461392bf4748bb1633205e738245', 'request_id': 'req-3fca1b99-a2c4-4c9d-bf4d-b5a743ae3e89'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:56.494657 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3fca1b99-a2c4-4c9d-bf4d-b5a743ae3e89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4e7461392bf4748bb1633205e738245', 'request_id': 'req-3fca1b99-a2c4-4c9d-bf4d-b5a743ae3e89'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:56.502234 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3fca1b99-a2c4-4c9d-bf4d-b5a743ae3e89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4e7461392bf4748bb1633205e738245', 'request_id': 'req-3fca1b99-a2c4-4c9d-bf4d-b5a743ae3e89'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:56.517370 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3fca1b99-a2c4-4c9d-bf4d-b5a743ae3e89 admin admin] The token cache is being invalidate because project e4e7461392bf4748bb1633205e738245 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:02:56.517665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3fca1b99-a2c4-4c9d-bf4d-b5a743ae3e89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:56.518326 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3fca1b99-a2c4-4c9d-bf4d-b5a743ae3e89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:02:56.519939 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1095/2163] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:02:56 2022] DELETE /identity/v3/projects/e4e7461392bf4748bb1633205e738245 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:56.526058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83acf1d-f342-489a-a440-f55574fd71cb None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:56.526552 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83acf1d-f342-489a-a440-f55574fd71cb None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:56.526710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83acf1d-f342-489a-a440-f55574fd71cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:56.544939 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-b83acf1d-f342-489a-a440-f55574fd71cb None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:56.546230 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-b83acf1d-f342-489a-a440-f55574fd71cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:56.611412 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1069/2164] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:56.619330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47a126-b004-4ca4-9c65-232344d19aab None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:56.619710 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47a126-b004-4ca4-9c65-232344d19aab None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:56.620001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47a126-b004-4ca4-9c65-232344d19aab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:56.627623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:56.643210 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-0f47a126-b004-4ca4-9c65-232344d19aab None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:56.646432 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-0f47a126-b004-4ca4-9c65-232344d19aab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:56.655998 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1096/2165] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:56.660569 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83acf1d-f342-489a-a440-f55574fd71cb None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:56.671519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83acf1d-f342-489a-a440-f55574fd71cb None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:56.677118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e42d7-c3e4-46e1-a22b-1f426aeb04b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:56.677858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e42d7-c3e4-46e1-a22b-1f426aeb04b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:56.678791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e42d7-c3e4-46e1-a22b-1f426aeb04b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:56.679243 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e42d7-c3e4-46e1-a22b-1f426aeb04b8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:56.680096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e42d7-c3e4-46e1-a22b-1f426aeb04b8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:02:56.680479 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e42d7-c3e4-46e1-a22b-1f426aeb04b8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:56.682487 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e42d7-c3e4-46e1-a22b-1f426aeb04b8 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:56.697454 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1070/2166] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:02:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:02:56.701667 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47a126-b004-4ca4-9c65-232344d19aab None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:56.716059 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47a126-b004-4ca4-9c65-232344d19aab None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:56.721929 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068989df-8b35-455d-bffc-2fefbd8507e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:56.722741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068989df-8b35-455d-bffc-2fefbd8507e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:56.723089 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068989df-8b35-455d-bffc-2fefbd8507e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:56.723900 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068989df-8b35-455d-bffc-2fefbd8507e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:56.724542 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068989df-8b35-455d-bffc-2fefbd8507e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:56.726902 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068989df-8b35-455d-bffc-2fefbd8507e8 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:56.761209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1097/2167] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:02:56 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:02:56.765750 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e42d7-c3e4-46e1-a22b-1f426aeb04b8 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:56.781455 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e42d7-c3e4-46e1-a22b-1f426aeb04b8 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:56.854800 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09a5b0-f5df-4103-ac6a-830f6aebca63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:56.855771 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09a5b0-f5df-4103-ac6a-830f6aebca63 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:56.856052 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09a5b0-f5df-4103-ac6a-830f6aebca63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:56.856335 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af09a5b0-f5df-4103-ac6a-830f6aebca63 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:56.857032 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af09a5b0-f5df-4103-ac6a-830f6aebca63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:56.858996 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af09a5b0-f5df-4103-ac6a-830f6aebca63 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:56.890141 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1071/2168] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:02:56 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:02:56.891630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068989df-8b35-455d-bffc-2fefbd8507e8 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:56.904036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068989df-8b35-455d-bffc-2fefbd8507e8 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:56.908800 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d758b-9d0b-447e-89fa-d741b0e27754 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:56.909498 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d758b-9d0b-447e-89fa-d741b0e27754 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:56.909788 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d758b-9d0b-447e-89fa-d741b0e27754 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:56.910061 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152d758b-9d0b-447e-89fa-d741b0e27754 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:56.910841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d758b-9d0b-447e-89fa-d741b0e27754 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:56.913528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152d758b-9d0b-447e-89fa-d741b0e27754 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:56.949701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1098/2169] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:02:56.954068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09a5b0-f5df-4103-ac6a-830f6aebca63 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:56.964973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af09a5b0-f5df-4103-ac6a-830f6aebca63 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:56.969636 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ddf98a-21b8-4e25-9382-92076e14065a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:56.970316 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddf98a-21b8-4e25-9382-92076e14065a admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:56.970635 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddf98a-21b8-4e25-9382-92076e14065a admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:56.970925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ddf98a-21b8-4e25-9382-92076e14065a admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:56.971663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ddf98a-21b8-4e25-9382-92076e14065a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:56.973688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ddf98a-21b8-4e25-9382-92076e14065a admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:56.989811 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d758b-9d0b-447e-89fa-d741b0e27754 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:56.990258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1072/2170] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:02:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:02:57.000892 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152d758b-9d0b-447e-89fa-d741b0e27754 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:57.012572 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efecfda8-863b-410f-a15b-1f7556be7d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:57.013661 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecfda8-863b-410f-a15b-1f7556be7d43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:57.014025 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecfda8-863b-410f-a15b-1f7556be7d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:57.014222 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efecfda8-863b-410f-a15b-1f7556be7d43 admin admin] PATH_INFO: `/v3/projects/0a93e4687e544ce8b0a4e1559b8a558b/users/8ca17d54889646e2905c38638a0ad54d/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:57.035925 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecfda8-863b-410f-a15b-1f7556be7d43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0a93e4687e544ce8b0a4e1559b8a558b, user_id=8ca17d54889646e2905c38638a0ad54d, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:57.039816 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efecfda8-863b-410f-a15b-1f7556be7d43 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:57.056577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa687d6c-34ef-45cc-a932-dbb28415957b None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:57.057291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa687d6c-34ef-45cc-a932-dbb28415957b None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:57.057627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa687d6c-34ef-45cc-a932-dbb28415957b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:57.073947 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1099/2171] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:02:56 2022] PUT /identity/v3/projects/0a93e4687e544ce8b0a4e1559b8a558b/users/8ca17d54889646e2905c38638a0ad54d/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:02:57.084118 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de05d17-9d19-467a-9391-14a9d1015c97 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:57.084777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de05d17-9d19-467a-9391-14a9d1015c97 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:57.084777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de05d17-9d19-467a-9391-14a9d1015c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:57.109601 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-fa687d6c-34ef-45cc-a932-dbb28415957b None None] MFA Rules not processed for user `69163c28aa75492aa01c70532bed15a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:57.110852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-fa687d6c-34ef-45cc-a932-dbb28415957b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:57.116335 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-8de05d17-9d19-467a-9391-14a9d1015c97 None None] MFA Rules not processed for user `8ca17d54889646e2905c38638a0ad54d`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:57.117545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-8de05d17-9d19-467a-9391-14a9d1015c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:57.126188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:02:57.200457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1073/2172] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:02:57.211378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa687d6c-34ef-45cc-a932-dbb28415957b None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:57.211378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-fa687d6c-34ef-45cc-a932-dbb28415957b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:57.220986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1100/2173] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:57 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:02:57.233952 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de05d17-9d19-467a-9391-14a9d1015c97 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:02:57.233952 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-8de05d17-9d19-467a-9391-14a9d1015c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:57.278100 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa687d6c-34ef-45cc-a932-dbb28415957b None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:57.281897 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1158f13-a368-48fc-b763-24166ae10ca2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:57.282554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1158f13-a368-48fc-b763-24166ae10ca2 service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:57.282876 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1158f13-a368-48fc-b763-24166ae10ca2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:57.283158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1158f13-a368-48fc-b763-24166ae10ca2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:57.292235 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1158f13-a368-48fc-b763-24166ae10ca2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:57.293666 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1158f13-a368-48fc-b763-24166ae10ca2 service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:57.298361 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de05d17-9d19-467a-9391-14a9d1015c97 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:02:57.301974 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4512dcf-84db-473b-9b77-7bcb8995a3bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:02:57.302653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4512dcf-84db-473b-9b77-7bcb8995a3bb service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:57.303459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4512dcf-84db-473b-9b77-7bcb8995a3bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:57.303635 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1074/2174] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:02:57.303795 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4512dcf-84db-473b-9b77-7bcb8995a3bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:57.312405 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4512dcf-84db-473b-9b77-7bcb8995a3bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:02:57.313866 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4512dcf-84db-473b-9b77-7bcb8995a3bb service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:02:57.323633 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1101/2175] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:02:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:02:58.664093 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04b64f-0631-48f7-b693-c53834d58564 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:02:58.664600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04b64f-0631-48f7-b693-c53834d58564 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:02:58.664600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d04b64f-0631-48f7-b693-c53834d58564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:02:58.683486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-6d04b64f-0631-48f7-b693-c53834d58564 None None] MFA Rules not processed for user `69163c28aa75492aa01c70532bed15a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:02:58.684735 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6d04b64f-0631-48f7-b693-c53834d58564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:02:58.691793 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1075/2176] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:02:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:03:04.047338 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af7f45-a439-4352-bc39-a82797ec2f5a None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:04.047780 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af7f45-a439-4352-bc39-a82797ec2f5a None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:04.048057 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42af7f45-a439-4352-bc39-a82797ec2f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:04.067213 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-42af7f45-a439-4352-bc39-a82797ec2f5a None None] MFA Rules not processed for user `9e3a2d16133b4af8a208682de8a7d111`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:03:04.068482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-42af7f45-a439-4352-bc39-a82797ec2f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:03:04.098657 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:03:04.147173 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1102/2177] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:03:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:03:08.006141 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04b64f-0631-48f7-b693-c53834d58564 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:03:08.152897 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d04b64f-0631-48f7-b693-c53834d58564 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:03:08.159914 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff7762-191b-4002-b9dc-703cb6643902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:03:08.160943 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff7762-191b-4002-b9dc-703cb6643902 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:08.161858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff7762-191b-4002-b9dc-703cb6643902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:08.162243 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aff7762-191b-4002-b9dc-703cb6643902 admin admin] PATH_INFO: `/v3/users/8ca17d54889646e2905c38638a0ad54d` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:08.171469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff7762-191b-4002-b9dc-703cb6643902 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ca17d54889646e2905c38638a0ad54d)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:03:08.175866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aff7762-191b-4002-b9dc-703cb6643902 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:03:08.196394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:03:08.274499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-3aff7762-191b-4002-b9dc-703cb6643902 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ca17d54889646e2905c38638a0ad54d'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:03:08.275238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-3aff7762-191b-4002-b9dc-703cb6643902 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ca17d54889646e2905c38638a0ad54d'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:03:08.282892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-3aff7762-191b-4002-b9dc-703cb6643902 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ca17d54889646e2905c38638a0ad54d'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:03:08.283722 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-3aff7762-191b-4002-b9dc-703cb6643902 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ca17d54889646e2905c38638a0ad54d'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:03:08.314800 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-3aff7762-191b-4002-b9dc-703cb6643902 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ca17d54889646e2905c38638a0ad54d'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:03:08.341429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1076/2178] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:03:08 2022] DELETE /identity/v3/users/8ca17d54889646e2905c38638a0ad54d => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:03:08.346000 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4badbe-ff52-4d25-9153-1b4888e3064b None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:08.346508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4badbe-ff52-4d25-9153-1b4888e3064b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:08.346912 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4badbe-ff52-4d25-9153-1b4888e3064b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:08.446979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-8d4badbe-ff52-4d25-9153-1b4888e3064b None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:03:08.448097 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-8d4badbe-ff52-4d25-9153-1b4888e3064b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:03:08.541262 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1103/2179] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:03:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:03:08.548924 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff7762-191b-4002-b9dc-703cb6643902 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:03:08.550468 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-3aff7762-191b-4002-b9dc-703cb6643902 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:03:08.697726 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aff7762-191b-4002-b9dc-703cb6643902 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:03:08.701348 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca20a39a-65fc-473b-bfbd-55050fa1d0e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:03:08.702007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20a39a-65fc-473b-bfbd-55050fa1d0e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:08.702292 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20a39a-65fc-473b-bfbd-55050fa1d0e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:08.702567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca20a39a-65fc-473b-bfbd-55050fa1d0e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:08.712061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20a39a-65fc-473b-bfbd-55050fa1d0e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:03:08.713612 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca20a39a-65fc-473b-bfbd-55050fa1d0e6 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:03:08.723591 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1077/2180] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:03:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:03:09.191091 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4badbe-ff52-4d25-9153-1b4888e3064b None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:03:09.192724 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-8d4badbe-ff52-4d25-9153-1b4888e3064b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:03:09.204918 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4badbe-ff52-4d25-9153-1b4888e3064b None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:03:09.214986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd88378-712e-45e3-bad6-4409d3cd307f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:03:09.215985 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd88378-712e-45e3-bad6-4409d3cd307f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:09.216273 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd88378-712e-45e3-bad6-4409d3cd307f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:09.216556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd88378-712e-45e3-bad6-4409d3cd307f admin admin] PATH_INFO: `/v3/projects/0a93e4687e544ce8b0a4e1559b8a558b` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:09.217786 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd88378-712e-45e3-bad6-4409d3cd307f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0a93e4687e544ce8b0a4e1559b8a558b)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:03:09.219877 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd88378-712e-45e3-bad6-4409d3cd307f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:03:09.270865 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-7bd88378-712e-45e3-bad6-4409d3cd307f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0a93e4687e544ce8b0a4e1559b8a558b', 'request_id': 'req-7bd88378-712e-45e3-bad6-4409d3cd307f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:03:09.277378 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-7bd88378-712e-45e3-bad6-4409d3cd307f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0a93e4687e544ce8b0a4e1559b8a558b', 'request_id': 'req-7bd88378-712e-45e3-bad6-4409d3cd307f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:03:09.283208 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-7bd88378-712e-45e3-bad6-4409d3cd307f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0a93e4687e544ce8b0a4e1559b8a558b', 'request_id': 'req-7bd88378-712e-45e3-bad6-4409d3cd307f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:03:09.299319 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-7bd88378-712e-45e3-bad6-4409d3cd307f admin admin] The token cache is being invalidate because project 0a93e4687e544ce8b0a4e1559b8a558b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:03:09.299655 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-7bd88378-712e-45e3-bad6-4409d3cd307f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:03:09.300267 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-7bd88378-712e-45e3-bad6-4409d3cd307f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:03:09.302853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1104/2181] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:03:09 2022] DELETE /identity/v3/projects/0a93e4687e544ce8b0a4e1559b8a558b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:03:15.199943 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4300a1-959f-4450-8dbb-70e1a4f6822d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:15.200865 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4300a1-959f-4450-8dbb-70e1a4f6822d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:15.201092 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4300a1-959f-4450-8dbb-70e1a4f6822d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:15.231992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-df4300a1-959f-4450-8dbb-70e1a4f6822d None None] MFA Rules not processed for user `faf778e713b54a638c1db4a5db6180b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:03:15.233806 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-df4300a1-959f-4450-8dbb-70e1a4f6822d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:03:15.327062 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:03:15.338762 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1078/2182] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 23:03:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:03:15.358719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd88378-712e-45e3-bad6-4409d3cd307f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:03:15.359489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7bd88378-712e-45e3-bad6-4409d3cd307f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:03:15.439174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:03:15.489701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd88378-712e-45e3-bad6-4409d3cd307f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:03:15.494735 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25dca91-4f11-472a-a3e2-6623f31829c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef518c9430e4274b1066241f821f4a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:03:15.495424 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25dca91-4f11-472a-a3e2-6623f31829c6 service swift] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:15.495713 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25dca91-4f11-472a-a3e2-6623f31829c6 service swift] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:15.495995 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25dca91-4f11-472a-a3e2-6623f31829c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:15.505113 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25dca91-4f11-472a-a3e2-6623f31829c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:03:15.506512 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25dca91-4f11-472a-a3e2-6623f31829c6 service swift] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:03:15.516406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1105/2183] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 23:03:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 23:03:25.520202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=16 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:03:29.642460 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ccc2d3-e78c-44a7-b8cd-ef648db46d55 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:29.643575 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ccc2d3-e78c-44a7-b8cd-ef648db46d55 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:29.643575 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ccc2d3-e78c-44a7-b8cd-ef648db46d55 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:29.644853 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1079/2184] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 23:03:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 23:03:29.650607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987c42c-99dc-4c63-b05d-7b00f53c604c None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:29.651096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987c42c-99dc-4c63-b05d-7b00f53c604c None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:29.651301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4987c42c-99dc-4c63-b05d-7b00f53c604c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:29.692518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-4987c42c-99dc-4c63-b05d-7b00f53c604c None None] MFA Rules not processed for user `4180f2593149437fa27effd8d1338d10`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:03:29.693923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-4987c42c-99dc-4c63-b05d-7b00f53c604c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:03:29.761461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1106/2185] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 23:03:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:03:29.773745 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ccc2d3-e78c-44a7-b8cd-ef648db46d55 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:03:29.775529 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c7ccc2d3-e78c-44a7-b8cd-ef648db46d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:03:29.835842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ccc2d3-e78c-44a7-b8cd-ef648db46d55 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:03:29.839411 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f5c3e4-1953-4eea-9874-59a4e8b9f9e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915f4296504b47d796e3a5b99550eb75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:03:29.839992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5c3e4-1953-4eea-9874-59a4e8b9f9e7 service glance] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:29.840277 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5c3e4-1953-4eea-9874-59a4e8b9f9e7 service glance] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:29.840551 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f5c3e4-1953-4eea-9874-59a4e8b9f9e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:29.851333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5c3e4-1953-4eea-9874-59a4e8b9f9e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:03:29.852758 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f5c3e4-1953-4eea-9874-59a4e8b9f9e7 service glance] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:03:29.863653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1080/2186] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:03:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:03:29.933198 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993279e-6016-4f10-bde3-bdd013536aac None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:29.933491 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993279e-6016-4f10-bde3-bdd013536aac None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:29.933775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8993279e-6016-4f10-bde3-bdd013536aac None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:29.935201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1107/2187] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 23:03:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 23:03:29.940261 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb2c0a-2621-450c-a538-e05c6a741c89 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:29.940576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb2c0a-2621-450c-a538-e05c6a741c89 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:29.940856 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bb2c0a-2621-450c-a538-e05c6a741c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:29.959486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-63bb2c0a-2621-450c-a538-e05c6a741c89 None None] MFA Rules not processed for user `6b794f7aa2a7488e91f7d9ea35a45d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:03:29.960675 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-63bb2c0a-2621-450c-a538-e05c6a741c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:03:30.013060 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1081/2188] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 23:03:29 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:03:30.024757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993279e-6016-4f10-bde3-bdd013536aac None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:03:30.026488 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-8993279e-6016-4f10-bde3-bdd013536aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:03:30.039951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8993279e-6016-4f10-bde3-bdd013536aac None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:03:30.043424 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e841ff2b-6ca6-4277-860a-7115844458e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:03:30.044060 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841ff2b-6ca6-4277-860a-7115844458e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:30.044342 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841ff2b-6ca6-4277-860a-7115844458e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:30.044618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e841ff2b-6ca6-4277-860a-7115844458e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:30.053033 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841ff2b-6ca6-4277-860a-7115844458e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:03:30.054638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e841ff2b-6ca6-4277-860a-7115844458e4 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:03:30.063648 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1108/2189] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:03:30 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:03:30.148059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f041049-fe62-4375-a980-42364adbdf93 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:30.148351 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f041049-fe62-4375-a980-42364adbdf93 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:30.148644 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f041049-fe62-4375-a980-42364adbdf93 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:30.150041 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1082/2190] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 23:03:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 23:03:30.155918 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d3f40-5326-48ec-b8c6-59a2c2770518 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:30.156215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d3f40-5326-48ec-b8c6-59a2c2770518 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:30.156423 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d3f40-5326-48ec-b8c6-59a2c2770518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:30.205093 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-ef8d3f40-5326-48ec-b8c6-59a2c2770518 None None] MFA Rules not processed for user `4180f2593149437fa27effd8d1338d10`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:03:30.206926 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ef8d3f40-5326-48ec-b8c6-59a2c2770518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:03:30.232524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1109/2191] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 23:03:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:03:30.242969 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f041049-fe62-4375-a980-42364adbdf93 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:03:30.244756 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-3f041049-fe62-4375-a980-42364adbdf93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:03:30.258651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f041049-fe62-4375-a980-42364adbdf93 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:03:30.262216 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa8723-ad91-432a-aba9-170a202e1d97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:03:30.262896 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa8723-ad91-432a-aba9-170a202e1d97 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:30.263182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa8723-ad91-432a-aba9-170a202e1d97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:30.263461 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa8723-ad91-432a-aba9-170a202e1d97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:30.273481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa8723-ad91-432a-aba9-170a202e1d97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:03:30.274941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa8723-ad91-432a-aba9-170a202e1d97 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:03:30.286656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1083/2192] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:03:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:03:32.787838 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5265f0c-4bdc-4169-8788-1563abdc0dc1 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:32.788309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5265f0c-4bdc-4169-8788-1563abdc0dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:32.788491 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5265f0c-4bdc-4169-8788-1563abdc0dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:32.806499 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-b5265f0c-4bdc-4169-8788-1563abdc0dc1 None None] MFA Rules not processed for user `faf778e713b54a638c1db4a5db6180b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:03:32.807749 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b5265f0c-4bdc-4169-8788-1563abdc0dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:03:32.815639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1110/2193] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 23:03:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:03:32.837501 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa8723-ad91-432a-aba9-170a202e1d97 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:03:32.850037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa8723-ad91-432a-aba9-170a202e1d97 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:03:32.855366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2812286-3d5b-49b0-9c80-8e44cce17e89 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef518c9430e4274b1066241f821f4a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:03:32.856007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2812286-3d5b-49b0-9c80-8e44cce17e89 service swift] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:32.856316 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2812286-3d5b-49b0-9c80-8e44cce17e89 service swift] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:32.856594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2812286-3d5b-49b0-9c80-8e44cce17e89 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:32.865383 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2812286-3d5b-49b0-9c80-8e44cce17e89 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:03:32.867009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2812286-3d5b-49b0-9c80-8e44cce17e89 service swift] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:03:32.875137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1084/2194] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 23:03:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 23:03:42.801244 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=69 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:03:42.871890 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=61 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:03:47.564909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5265f0c-4bdc-4169-8788-1563abdc0dc1 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:03:47.566464 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b5265f0c-4bdc-4169-8788-1563abdc0dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:03:47.660010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5265f0c-4bdc-4169-8788-1563abdc0dc1 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:03:47.665096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d61619-56b4-451d-bec0-fb2c02835f66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9593939641c4c7797c1981206b40e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:03:47.665739 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d61619-56b4-451d-bec0-fb2c02835f66 service placement] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:03:47.666022 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d61619-56b4-451d-bec0-fb2c02835f66 service placement] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:03:47.666298 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d61619-56b4-451d-bec0-fb2c02835f66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:03:47.668048 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-82d61619-56b4-451d-bec0-fb2c02835f66 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:03:47.680058 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d61619-56b4-451d-bec0-fb2c02835f66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:03:47.681606 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d61619-56b4-451d-bec0-fb2c02835f66 service placement] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:03:47.691785 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1111/2195] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:03:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:03:57.687054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=41 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:04:03.298505 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494aa00-0ad9-46a2-a45a-300a94c0c695 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:04:03.299123 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494aa00-0ad9-46a2-a45a-300a94c0c695 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:04:03.299311 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7494aa00-0ad9-46a2-a45a-300a94c0c695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:04:03.348594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-7494aa00-0ad9-46a2-a45a-300a94c0c695 None None] MFA Rules not processed for user `faf778e713b54a638c1db4a5db6180b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:04:03.349999 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-7494aa00-0ad9-46a2-a45a-300a94c0c695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:04:03.367651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1085/2196] 10.222.0.17 () {58 vars in 967 bytes} [Thu Jun 16 23:04:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:04:03.380966 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d61619-56b4-451d-bec0-fb2c02835f66 service placement] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:04:03.401078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d61619-56b4-451d-bec0-fb2c02835f66 service placement] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:04:03.406105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ddf850-11a6-4589-832f-35b2b398bddf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef518c9430e4274b1066241f821f4a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:04:03.406774 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddf850-11a6-4589-832f-35b2b398bddf service swift] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:04:03.407059 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddf850-11a6-4589-832f-35b2b398bddf service swift] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:04:03.407338 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ddf850-11a6-4589-832f-35b2b398bddf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:04:03.415286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddf850-11a6-4589-832f-35b2b398bddf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:04:03.416839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ddf850-11a6-4589-832f-35b2b398bddf service swift] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:04:03.425244 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1112/2197] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 23:04:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 23:04:13.364793 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=18 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:04:13.421471 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=10 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:04:18.696437 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af64a7-3761-4aed-8f1e-f6e16dcca005 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:04:18.696947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af64a7-3761-4aed-8f1e-f6e16dcca005 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:04:18.697062 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49af64a7-3761-4aed-8f1e-f6e16dcca005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:04:18.715586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-49af64a7-3761-4aed-8f1e-f6e16dcca005 None None] MFA Rules not processed for user `69163c28aa75492aa01c70532bed15a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:04:18.717948 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-49af64a7-3761-4aed-8f1e-f6e16dcca005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:04:18.773540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1086/2198] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:04:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:04:28.766814 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=25 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:04:33.526021 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dce4bd-dded-40c4-8e8f-273783ee68c0 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:04:33.526547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dce4bd-dded-40c4-8e8f-273783ee68c0 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:04:33.526775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dce4bd-dded-40c4-8e8f-273783ee68c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:04:33.545099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-23dce4bd-dded-40c4-8e8f-273783ee68c0 None None] MFA Rules not processed for user `243e592fbf1844d89f55dd5035a684f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:04:33.546253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-23dce4bd-dded-40c4-8e8f-273783ee68c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:04:33.600671 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1113/2199] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:04:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:04:43.593641 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=25 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:04:54.178262 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f2892-6614-4e66-a902-ceff623c0448 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:04:54.178803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f2892-6614-4e66-a902-ceff623c0448 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:04:54.178895 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92f2892-6614-4e66-a902-ceff623c0448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:04:54.196652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-d92f2892-6614-4e66-a902-ceff623c0448 None None] MFA Rules not processed for user `9e3a2d16133b4af8a208682de8a7d111`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:04:54.198885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d92f2892-6614-4e66-a902-ceff623c0448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:04:54.273100 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1087/2200] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:04:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:05:04.264877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=25 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:05:06.036159 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867ee52-f575-47df-94aa-5f2e42e3f35f None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:06.036673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867ee52-f575-47df-94aa-5f2e42e3f35f None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:06.036767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2867ee52-f575-47df-94aa-5f2e42e3f35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:06.054434 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-2867ee52-f575-47df-94aa-5f2e42e3f35f None None] MFA Rules not processed for user `a1b1a557a3834b98a3397c41665b5dc6`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:05:06.059962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-2867ee52-f575-47df-94aa-5f2e42e3f35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:06.118539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1114/2201] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:05:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:05:16.111181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=25 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:05:21.938330 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f2892-6614-4e66-a902-ceff623c0448 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:21.941381 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d92f2892-6614-4e66-a902-ceff623c0448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:22.009797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92f2892-6614-4e66-a902-ceff623c0448 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:22.014566 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440bb430-9b09-4abf-8eaf-9e05ff216ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:22.015862 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bb430-9b09-4abf-8eaf-9e05ff216ae7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:22.015862 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bb430-9b09-4abf-8eaf-9e05ff216ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:22.016075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440bb430-9b09-4abf-8eaf-9e05ff216ae7 admin admin] PATH_INFO: `/v3/users/69163c28aa75492aa01c70532bed15a9` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:22.017322 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440bb430-9b09-4abf-8eaf-9e05ff216ae7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=69163c28aa75492aa01c70532bed15a9)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:22.019245 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440bb430-9b09-4abf-8eaf-9e05ff216ae7 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:22.081908 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-440bb430-9b09-4abf-8eaf-9e05ff216ae7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69163c28aa75492aa01c70532bed15a9'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:22.085354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-440bb430-9b09-4abf-8eaf-9e05ff216ae7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69163c28aa75492aa01c70532bed15a9'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:22.095603 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-440bb430-9b09-4abf-8eaf-9e05ff216ae7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69163c28aa75492aa01c70532bed15a9'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:22.096739 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-440bb430-9b09-4abf-8eaf-9e05ff216ae7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69163c28aa75492aa01c70532bed15a9'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:22.116118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-440bb430-9b09-4abf-8eaf-9e05ff216ae7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69163c28aa75492aa01c70532bed15a9'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:22.138765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1088/2202] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:05:21 2022] DELETE /identity/v3/users/69163c28aa75492aa01c70532bed15a9 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:05:22.352278 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867ee52-f575-47df-94aa-5f2e42e3f35f None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:22.366017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2867ee52-f575-47df-94aa-5f2e42e3f35f None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:22.435700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdde36a-308c-4102-a6a9-be692ae7ba4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:22.436583 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdde36a-308c-4102-a6a9-be692ae7ba4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:22.436869 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdde36a-308c-4102-a6a9-be692ae7ba4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:22.437332 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdde36a-308c-4102-a6a9-be692ae7ba4b admin admin] PATH_INFO: `/v3/projects/9761759a69724a21b0b9315a931b106f` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:22.438567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdde36a-308c-4102-a6a9-be692ae7ba4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9761759a69724a21b0b9315a931b106f)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:22.440520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdde36a-308c-4102-a6a9-be692ae7ba4b admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:22.490772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-4bdde36a-308c-4102-a6a9-be692ae7ba4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9761759a69724a21b0b9315a931b106f', 'request_id': 'req-4bdde36a-308c-4102-a6a9-be692ae7ba4b'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:22.496955 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-4bdde36a-308c-4102-a6a9-be692ae7ba4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9761759a69724a21b0b9315a931b106f', 'request_id': 'req-4bdde36a-308c-4102-a6a9-be692ae7ba4b'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:22.502482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-4bdde36a-308c-4102-a6a9-be692ae7ba4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9761759a69724a21b0b9315a931b106f', 'request_id': 'req-4bdde36a-308c-4102-a6a9-be692ae7ba4b'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:22.516358 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-4bdde36a-308c-4102-a6a9-be692ae7ba4b admin admin] The token cache is being invalidate because project 9761759a69724a21b0b9315a931b106f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:05:22.516652 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-4bdde36a-308c-4102-a6a9-be692ae7ba4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:22.517334 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-4bdde36a-308c-4102-a6a9-be692ae7ba4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:22.518879 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1115/2203] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:05:22 2022] DELETE /identity/v3/projects/9761759a69724a21b0b9315a931b106f => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:05:22.524340 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5ce03-6592-4bdb-8b84-92a65e92557d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:22.524639 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5ce03-6592-4bdb-8b84-92a65e92557d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:22.524926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e5ce03-6592-4bdb-8b84-92a65e92557d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:22.566496 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-36e5ce03-6592-4bdb-8b84-92a65e92557d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:05:22.567733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-36e5ce03-6592-4bdb-8b84-92a65e92557d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:22.628141 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1089/2204] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:05:22.636219 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ee0c5-f076-41da-bfb3-ce4402a92ace None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:22.636509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ee0c5-f076-41da-bfb3-ce4402a92ace None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:22.636819 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ee0c5-f076-41da-bfb3-ce4402a92ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:22.657532 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-f69ee0c5-f076-41da-bfb3-ce4402a92ace None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:05:22.658777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-f69ee0c5-f076-41da-bfb3-ce4402a92ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:22.666751 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1116/2205] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:05:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:05:22.671066 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5ce03-6592-4bdb-8b84-92a65e92557d None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:22.677675 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:05:22.680826 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e5ce03-6592-4bdb-8b84-92a65e92557d None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:22.687036 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c8a93-3bcb-4357-aabf-1fe75772801f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:22.688029 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c8a93-3bcb-4357-aabf-1fe75772801f admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:22.688496 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c8a93-3bcb-4357-aabf-1fe75772801f admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:22.688927 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c8a93-3bcb-4357-aabf-1fe75772801f admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:22.689762 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c8a93-3bcb-4357-aabf-1fe75772801f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:05:22.690333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c8a93-3bcb-4357-aabf-1fe75772801f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:22.692193 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c8a93-3bcb-4357-aabf-1fe75772801f admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:22.707323 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1090/2206] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:05:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:05:22.711618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ee0c5-f076-41da-bfb3-ce4402a92ace None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:22.721256 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ee0c5-f076-41da-bfb3-ce4402a92ace None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:22.730776 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9499f-1b33-4e95-a4e5-84f416f81650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:22.732849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9499f-1b33-4e95-a4e5-84f416f81650 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:22.733286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9499f-1b33-4e95-a4e5-84f416f81650 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:22.733573 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9499f-1b33-4e95-a4e5-84f416f81650 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:22.734274 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9499f-1b33-4e95-a4e5-84f416f81650 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:22.740162 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9499f-1b33-4e95-a4e5-84f416f81650 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:22.778371 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c8a93-3bcb-4357-aabf-1fe75772801f admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:22.778576 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1117/2207] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:05:22 2022] POST /identity/v3/projects => generated 391 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:05:22.789108 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c8a93-3bcb-4357-aabf-1fe75772801f admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:22.848155 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048111e-9bc8-4611-84be-efbb4efbeff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:22.848833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048111e-9bc8-4611-84be-efbb4efbeff1 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:22.849120 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048111e-9bc8-4611-84be-efbb4efbeff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:22.849399 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0048111e-9bc8-4611-84be-efbb4efbeff1 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:22.850134 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0048111e-9bc8-4611-84be-efbb4efbeff1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:22.852076 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0048111e-9bc8-4611-84be-efbb4efbeff1 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:22.873983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1091/2208] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:05:22 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:05:22.877977 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9499f-1b33-4e95-a4e5-84f416f81650 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:22.888745 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9499f-1b33-4e95-a4e5-84f416f81650 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:22.893341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ad924-ab37-461d-aa81-a3c9fdad4979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:22.894008 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ad924-ab37-461d-aa81-a3c9fdad4979 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:22.894292 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ad924-ab37-461d-aa81-a3c9fdad4979 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:22.894646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16ad924-ab37-461d-aa81-a3c9fdad4979 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:22.895574 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ad924-ab37-461d-aa81-a3c9fdad4979 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:22.897222 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16ad924-ab37-461d-aa81-a3c9fdad4979 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:22.908209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1118/2209] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:05:22.911908 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048111e-9bc8-4611-84be-efbb4efbeff1 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:22.922342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0048111e-9bc8-4611-84be-efbb4efbeff1 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:22.927221 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecf6a87-ed6a-4c23-94f5-13b9dc5055e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:22.928467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf6a87-ed6a-4c23-94f5-13b9dc5055e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:22.928756 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf6a87-ed6a-4c23-94f5-13b9dc5055e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:22.929038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecf6a87-ed6a-4c23-94f5-13b9dc5055e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:22.929764 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf6a87-ed6a-4c23-94f5-13b9dc5055e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:22.931446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ecf6a87-ed6a-4c23-94f5-13b9dc5055e3 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:22.945250 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1092/2210] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:05:22 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:05:22.946343 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ad924-ab37-461d-aa81-a3c9fdad4979 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:22.956292 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16ad924-ab37-461d-aa81-a3c9fdad4979 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:22.961133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc797c3-fa6c-4335-a3ce-20c1b2dcc376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:22.962008 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc797c3-fa6c-4335-a3ce-20c1b2dcc376 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:22.962292 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc797c3-fa6c-4335-a3ce-20c1b2dcc376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:22.962570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc797c3-fa6c-4335-a3ce-20c1b2dcc376 admin admin] PATH_INFO: `/v3/projects/00eb0e763b034fa581e7f1e96a6e571f/users/e186fa250f32458b9d9ba383ea290723/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:22.976832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc797c3-fa6c-4335-a3ce-20c1b2dcc376 admin admin] RBAC: Authorizing `identity:create_grant(project_id=00eb0e763b034fa581e7f1e96a6e571f, user_id=e186fa250f32458b9d9ba383ea290723, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:22.979521 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc797c3-fa6c-4335-a3ce-20c1b2dcc376 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:22.994511 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1119/2211] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:05:22 2022] PUT /identity/v3/projects/00eb0e763b034fa581e7f1e96a6e571f/users/e186fa250f32458b9d9ba383ea290723/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:05:23.004794 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50aa94-0619-4e8b-9f69-62a98e4d45e0 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:23.005085 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50aa94-0619-4e8b-9f69-62a98e4d45e0 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:23.005364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b50aa94-0619-4e8b-9f69-62a98e4d45e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:23.031819 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-3b50aa94-0619-4e8b-9f69-62a98e4d45e0 None None] MFA Rules not processed for user `e186fa250f32458b9d9ba383ea290723`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:05:23.032955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-3b50aa94-0619-4e8b-9f69-62a98e4d45e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:23.110902 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1093/2212] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:05:23 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:05:23.122560 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc797c3-fa6c-4335-a3ce-20c1b2dcc376 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:23.124681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-8bc797c3-fa6c-4335-a3ce-20c1b2dcc376 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:23.159044 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:05:23.221482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc797c3-fa6c-4335-a3ce-20c1b2dcc376 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:23.226025 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769f0b2-8127-4ac2-b26f-14805d7823eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:23.226738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769f0b2-8127-4ac2-b26f-14805d7823eb service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:23.227029 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769f0b2-8127-4ac2-b26f-14805d7823eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:23.227312 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5769f0b2-8127-4ac2-b26f-14805d7823eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:23.243722 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769f0b2-8127-4ac2-b26f-14805d7823eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:23.245270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5769f0b2-8127-4ac2-b26f-14805d7823eb service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:23.256463 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1120/2213] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:05:23 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:05:33.107998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=70,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:05:33.251918 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=23 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:05:40.776993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50aa94-0619-4e8b-9f69-62a98e4d45e0 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:40.780148 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-3b50aa94-0619-4e8b-9f69-62a98e4d45e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:40.854183 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b50aa94-0619-4e8b-9f69-62a98e4d45e0 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:40.857672 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf85d5-4ee3-49b0-99f4-10d301d4a606 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:40.858321 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf85d5-4ee3-49b0-99f4-10d301d4a606 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:40.858622 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf85d5-4ee3-49b0-99f4-10d301d4a606 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:40.858911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf85d5-4ee3-49b0-99f4-10d301d4a606 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:40.860683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-fcdf85d5-4ee3-49b0-99f4-10d301d4a606 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:40.935576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf85d5-4ee3-49b0-99f4-10d301d4a606 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:40.937056 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf85d5-4ee3-49b0-99f4-10d301d4a606 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:40.952311 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1094/2214] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:05:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:05:42.789704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769f0b2-8127-4ac2-b26f-14805d7823eb service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:42.791261 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5769f0b2-8127-4ac2-b26f-14805d7823eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:42.810128 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf85d5-4ee3-49b0-99f4-10d301d4a606 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:42.812003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5769f0b2-8127-4ac2-b26f-14805d7823eb service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:42.818491 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317dacbc-1604-4654-abc5-c7c318e9adc7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:42.819310 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317dacbc-1604-4654-abc5-c7c318e9adc7 service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:42.819598 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317dacbc-1604-4654-abc5-c7c318e9adc7 service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:42.819899 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317dacbc-1604-4654-abc5-c7c318e9adc7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:42.821911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-317dacbc-1604-4654-abc5-c7c318e9adc7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:42.886807 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf85d5-4ee3-49b0-99f4-10d301d4a606 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:42.891046 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317dacbc-1604-4654-abc5-c7c318e9adc7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:42.891607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde6c2d-f0cb-4280-a5fa-c4eec0085c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:42.892493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde6c2d-f0cb-4280-a5fa-c4eec0085c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:42.892777 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde6c2d-f0cb-4280-a5fa-c4eec0085c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:42.893057 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efde6c2d-f0cb-4280-a5fa-c4eec0085c76 admin admin] PATH_INFO: `/v3/users/e186fa250f32458b9d9ba383ea290723` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:42.893339 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317dacbc-1604-4654-abc5-c7c318e9adc7 service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:42.894297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde6c2d-f0cb-4280-a5fa-c4eec0085c76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e186fa250f32458b9d9ba383ea290723)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:42.896303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efde6c2d-f0cb-4280-a5fa-c4eec0085c76 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:42.904690 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1121/2215] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:05:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:05:42.946284 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:05:42.948981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-efde6c2d-f0cb-4280-a5fa-c4eec0085c76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e186fa250f32458b9d9ba383ea290723'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:42.949786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-efde6c2d-f0cb-4280-a5fa-c4eec0085c76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e186fa250f32458b9d9ba383ea290723'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:42.965464 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-efde6c2d-f0cb-4280-a5fa-c4eec0085c76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e186fa250f32458b9d9ba383ea290723'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:42.966221 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-efde6c2d-f0cb-4280-a5fa-c4eec0085c76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e186fa250f32458b9d9ba383ea290723'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:42.983430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-efde6c2d-f0cb-4280-a5fa-c4eec0085c76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e186fa250f32458b9d9ba383ea290723'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:43.002494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1095/2216] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:05:42 2022] DELETE /identity/v3/users/e186fa250f32458b9d9ba383ea290723 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:05:43.004455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12530fe3-b805-46ba-ba4d-1bfe6dbc186e None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:43.004769 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12530fe3-b805-46ba-ba4d-1bfe6dbc186e None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:43.005091 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12530fe3-b805-46ba-ba4d-1bfe6dbc186e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:43.036873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-12530fe3-b805-46ba-ba4d-1bfe6dbc186e None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:05:43.038342 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-12530fe3-b805-46ba-ba4d-1bfe6dbc186e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:43.116013 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1122/2217] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:05:43.135611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde6c2d-f0cb-4280-a5fa-c4eec0085c76 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:43.137897 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-efde6c2d-f0cb-4280-a5fa-c4eec0085c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:43.207466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efde6c2d-f0cb-4280-a5fa-c4eec0085c76 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:43.212923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee43e97-30bb-4115-9ff2-546f1fe79c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:43.213870 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee43e97-30bb-4115-9ff2-546f1fe79c39 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:43.214205 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee43e97-30bb-4115-9ff2-546f1fe79c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:43.214494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee43e97-30bb-4115-9ff2-546f1fe79c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:43.225930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee43e97-30bb-4115-9ff2-546f1fe79c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:43.227396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee43e97-30bb-4115-9ff2-546f1fe79c39 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:43.238178 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1096/2218] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:05:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:05:43.575199 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12530fe3-b805-46ba-ba4d-1bfe6dbc186e None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:43.577077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-12530fe3-b805-46ba-ba4d-1bfe6dbc186e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:43.601405 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12530fe3-b805-46ba-ba4d-1bfe6dbc186e None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:43.607161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7eef13-5cd1-45ba-b75a-09a7b34fa00e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:43.608188 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7eef13-5cd1-45ba-b75a-09a7b34fa00e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:43.608547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7eef13-5cd1-45ba-b75a-09a7b34fa00e admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:43.609419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7eef13-5cd1-45ba-b75a-09a7b34fa00e admin admin] PATH_INFO: `/v3/projects/00eb0e763b034fa581e7f1e96a6e571f` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:43.610557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7eef13-5cd1-45ba-b75a-09a7b34fa00e admin admin] RBAC: Authorizing `identity:delete_project(project_id=00eb0e763b034fa581e7f1e96a6e571f)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:43.612547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7eef13-5cd1-45ba-b75a-09a7b34fa00e admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:43.687310 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-7f7eef13-5cd1-45ba-b75a-09a7b34fa00e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00eb0e763b034fa581e7f1e96a6e571f', 'request_id': 'req-7f7eef13-5cd1-45ba-b75a-09a7b34fa00e'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:43.704085 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-7f7eef13-5cd1-45ba-b75a-09a7b34fa00e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00eb0e763b034fa581e7f1e96a6e571f', 'request_id': 'req-7f7eef13-5cd1-45ba-b75a-09a7b34fa00e'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:43.718082 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-7f7eef13-5cd1-45ba-b75a-09a7b34fa00e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00eb0e763b034fa581e7f1e96a6e571f', 'request_id': 'req-7f7eef13-5cd1-45ba-b75a-09a7b34fa00e'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:43.737040 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-7f7eef13-5cd1-45ba-b75a-09a7b34fa00e admin admin] The token cache is being invalidate because project 00eb0e763b034fa581e7f1e96a6e571f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:05:43.737337 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-7f7eef13-5cd1-45ba-b75a-09a7b34fa00e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:43.738001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-7f7eef13-5cd1-45ba-b75a-09a7b34fa00e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:43.740237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1123/2219] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:05:43 2022] DELETE /identity/v3/projects/00eb0e763b034fa581e7f1e96a6e571f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:05:43.746194 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f702e7e-ede2-41fd-a833-8da0e141bb95 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:43.746500 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f702e7e-ede2-41fd-a833-8da0e141bb95 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:43.746791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f702e7e-ede2-41fd-a833-8da0e141bb95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:43.765484 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-3f702e7e-ede2-41fd-a833-8da0e141bb95 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:05:43.766721 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-3f702e7e-ede2-41fd-a833-8da0e141bb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:43.780271 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7eef13-5cd1-45ba-b75a-09a7b34fa00e admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:43.781953 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7f7eef13-5cd1-45ba-b75a-09a7b34fa00e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:43.811863 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:05:43.839355 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1097/2220] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:05:43.847776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255a50d-cfcb-4195-ac0b-a54c37db4cc2 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:43.848068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255a50d-cfcb-4195-ac0b-a54c37db4cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:43.848359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255a50d-cfcb-4195-ac0b-a54c37db4cc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:43.869274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-5255a50d-cfcb-4195-ac0b-a54c37db4cc2 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:05:43.870527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-5255a50d-cfcb-4195-ac0b-a54c37db4cc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:43.876196 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7eef13-5cd1-45ba-b75a-09a7b34fa00e admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:43.879303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1098/2221] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:05:43.886055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255a50d-cfcb-4195-ac0b-a54c37db4cc2 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:43.888856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54392491-a8b8-4cd6-9f09-5d11adc631cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:43.889720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54392491-a8b8-4cd6-9f09-5d11adc631cd service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:43.890335 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54392491-a8b8-4cd6-9f09-5d11adc631cd service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:43.890626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54392491-a8b8-4cd6-9f09-5d11adc631cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:43.894170 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-54392491-a8b8-4cd6-9f09-5d11adc631cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:43.910683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255a50d-cfcb-4195-ac0b-a54c37db4cc2 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:43.916799 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb4c0f-e6b0-4d26-93af-7836a06ead71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:43.917822 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb4c0f-e6b0-4d26-93af-7836a06ead71 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:43.918315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb4c0f-e6b0-4d26-93af-7836a06ead71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:43.918826 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb4c0f-e6b0-4d26-93af-7836a06ead71 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:43.919947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb4c0f-e6b0-4d26-93af-7836a06ead71 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:05:43.920468 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb4c0f-e6b0-4d26-93af-7836a06ead71 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:43.922725 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb4c0f-e6b0-4d26-93af-7836a06ead71 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:43.940583 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1099/2222] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:05:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:05:43.942965 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb4c0f-e6b0-4d26-93af-7836a06ead71 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:43.959154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb4c0f-e6b0-4d26-93af-7836a06ead71 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:43.965545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3507951-7e75-4134-a3fa-4e34693e4aff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:43.966357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3507951-7e75-4134-a3fa-4e34693e4aff admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:43.966494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3507951-7e75-4134-a3fa-4e34693e4aff admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:43.967014 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3507951-7e75-4134-a3fa-4e34693e4aff admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:43.967717 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3507951-7e75-4134-a3fa-4e34693e4aff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:43.969986 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3507951-7e75-4134-a3fa-4e34693e4aff admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:43.986617 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54392491-a8b8-4cd6-9f09-5d11adc631cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:43.988262 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54392491-a8b8-4cd6-9f09-5d11adc631cd service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:43.999012 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1124/2223] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:05:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:05:44.012616 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1100/2224] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:05:43 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:05:44.016448 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54392491-a8b8-4cd6-9f09-5d11adc631cd service nova] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:44.028428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54392491-a8b8-4cd6-9f09-5d11adc631cd service nova] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:44.101992 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8060453-413b-4db0-aa97-da98caa48606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:44.102750 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8060453-413b-4db0-aa97-da98caa48606 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:44.103045 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8060453-413b-4db0-aa97-da98caa48606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:44.103686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8060453-413b-4db0-aa97-da98caa48606 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:44.104399 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8060453-413b-4db0-aa97-da98caa48606 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:44.106366 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8060453-413b-4db0-aa97-da98caa48606 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:44.134377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1125/2225] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:05:44 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:05:44.135510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3507951-7e75-4134-a3fa-4e34693e4aff admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:44.146124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3507951-7e75-4134-a3fa-4e34693e4aff admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:44.152177 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a2193-70a6-40c6-817b-5c5b386edc46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:44.152861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a2193-70a6-40c6-817b-5c5b386edc46 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:44.153003 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a2193-70a6-40c6-817b-5c5b386edc46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:44.153226 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786a2193-70a6-40c6-817b-5c5b386edc46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:44.154344 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a2193-70a6-40c6-817b-5c5b386edc46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:44.155597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786a2193-70a6-40c6-817b-5c5b386edc46 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:44.175246 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1101/2226] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:05:44.176498 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8060453-413b-4db0-aa97-da98caa48606 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:44.188635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8060453-413b-4db0-aa97-da98caa48606 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:44.195564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9dcec1-ccad-419c-aeb9-e902dc61085c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:44.196322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dcec1-ccad-419c-aeb9-e902dc61085c admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:44.196610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dcec1-ccad-419c-aeb9-e902dc61085c admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:44.196888 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9dcec1-ccad-419c-aeb9-e902dc61085c admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:44.197618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9dcec1-ccad-419c-aeb9-e902dc61085c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:44.199268 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9dcec1-ccad-419c-aeb9-e902dc61085c admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:44.226074 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1126/2227] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:05:44 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:05:44.227732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a2193-70a6-40c6-817b-5c5b386edc46 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:44.243813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786a2193-70a6-40c6-817b-5c5b386edc46 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:44.250977 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f28b4c-c296-496a-83b7-3a8cfcec6bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:44.251882 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f28b4c-c296-496a-83b7-3a8cfcec6bee admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:44.252181 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f28b4c-c296-496a-83b7-3a8cfcec6bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:44.252461 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f28b4c-c296-496a-83b7-3a8cfcec6bee admin admin] PATH_INFO: `/v3/projects/1423c05c42b14519b401e22e16062781/users/2697aa77265e4f5783f87ffb3d4992b5/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:44.264764 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:DELETE=4,keystone:SELECT=94,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:05:44.274427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f28b4c-c296-496a-83b7-3a8cfcec6bee admin admin] RBAC: Authorizing `identity:create_grant(project_id=1423c05c42b14519b401e22e16062781, user_id=2697aa77265e4f5783f87ffb3d4992b5, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:44.277270 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f28b4c-c296-496a-83b7-3a8cfcec6bee admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:44.301080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1102/2228] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:05:44 2022] PUT /identity/v3/projects/1423c05c42b14519b401e22e16062781/users/2697aa77265e4f5783f87ffb3d4992b5/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:05:44.304222 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382eea7-2d22-4d62-af64-612a24ecdd4e None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:44.304517 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382eea7-2d22-4d62-af64-612a24ecdd4e None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:44.304801 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382eea7-2d22-4d62-af64-612a24ecdd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:44.326346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-d382eea7-2d22-4d62-af64-612a24ecdd4e None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:05:44.328144 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-d382eea7-2d22-4d62-af64-612a24ecdd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:44.371083 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:05:44.403407 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1127/2229] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:05:44.413908 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f28b4c-c296-496a-83b7-3a8cfcec6bee admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:44.415803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-b2f28b4c-c296-496a-83b7-3a8cfcec6bee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:44.476564 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f28b4c-c296-496a-83b7-3a8cfcec6bee admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:44.480261 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee793f-c3d9-4525-b7c7-b83aeecc30fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:44.480917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee793f-c3d9-4525-b7c7-b83aeecc30fe service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:44.481204 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee793f-c3d9-4525-b7c7-b83aeecc30fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:44.481484 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcee793f-c3d9-4525-b7c7-b83aeecc30fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:44.491251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee793f-c3d9-4525-b7c7-b83aeecc30fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:44.492893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcee793f-c3d9-4525-b7c7-b83aeecc30fe service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:44.502958 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1103/2230] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:05:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:05:44.573262 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410734f-828f-4663-b650-4ca440b5e419 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:44.573592 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410734f-828f-4663-b650-4ca440b5e419 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:44.573832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7410734f-828f-4663-b650-4ca440b5e419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:44.594989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-7410734f-828f-4663-b650-4ca440b5e419 None None] MFA Rules not processed for user `6a993f5d14b64fccad723abecfcadf90`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:05:44.596720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7410734f-828f-4663-b650-4ca440b5e419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:44.654461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1128/2231] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:05:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:05:44.694004 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee793f-c3d9-4525-b7c7-b83aeecc30fe service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:44.695550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-dcee793f-c3d9-4525-b7c7-b83aeecc30fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:44.781779 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcee793f-c3d9-4525-b7c7-b83aeecc30fe service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:44.785552 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f19c66-183d-494b-a1ce-7b45b23f1750 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915f4296504b47d796e3a5b99550eb75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:44.786843 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f19c66-183d-494b-a1ce-7b45b23f1750 service glance] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:44.787161 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f19c66-183d-494b-a1ce-7b45b23f1750 service glance] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:44.787448 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f19c66-183d-494b-a1ce-7b45b23f1750 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:44.789232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-b7f19c66-183d-494b-a1ce-7b45b23f1750 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:44.872113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f19c66-183d-494b-a1ce-7b45b23f1750 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:44.873754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f19c66-183d-494b-a1ce-7b45b23f1750 service glance] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:44.886094 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1104/2232] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:05:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:05:46.017979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410734f-828f-4663-b650-4ca440b5e419 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:46.019584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7410734f-828f-4663-b650-4ca440b5e419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:46.119400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7410734f-828f-4663-b650-4ca440b5e419 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:46.125436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34837369-3c43-46e2-8f7d-945c3cb80a90 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9593939641c4c7797c1981206b40e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:46.126104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34837369-3c43-46e2-8f7d-945c3cb80a90 service placement] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:46.126392 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34837369-3c43-46e2-8f7d-945c3cb80a90 service placement] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:46.126703 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34837369-3c43-46e2-8f7d-945c3cb80a90 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:46.129016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-34837369-3c43-46e2-8f7d-945c3cb80a90 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:46.142525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34837369-3c43-46e2-8f7d-945c3cb80a90 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:46.144175 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34837369-3c43-46e2-8f7d-945c3cb80a90 service placement] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:46.155058 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1129/2233] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:05:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:05:50.203891 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25a657-27cf-48cc-a789-79c63af8e5c2 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:50.204348 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25a657-27cf-48cc-a789-79c63af8e5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:50.204452 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c25a657-27cf-48cc-a789-79c63af8e5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:50.240096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:05:50.253450 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-2c25a657-27cf-48cc-a789-79c63af8e5c2 None None] MFA Rules not processed for user `2697aa77265e4f5783f87ffb3d4992b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:05:50.255465 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-2c25a657-27cf-48cc-a789-79c63af8e5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:50.335717 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34837369-3c43-46e2-8f7d-945c3cb80a90 service placement] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:50.348538 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1105/2234] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:05:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:05:50.352693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34837369-3c43-46e2-8f7d-945c3cb80a90 service placement] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:50.352877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25a657-27cf-48cc-a789-79c63af8e5c2 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:50.360055 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e1165-b4cd-4069-bcc9-d51344497fc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:50.360833 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e1165-b4cd-4069-bcc9-d51344497fc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:50.361182 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e1165-b4cd-4069-bcc9-d51344497fc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:50.361531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9e1165-b4cd-4069-bcc9-d51344497fc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:50.366674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-0a9e1165-b4cd-4069-bcc9-d51344497fc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:50.372372 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c25a657-27cf-48cc-a789-79c63af8e5c2 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:50.377552 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065d245-bfe3-48c1-972a-82476efe0be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:50.378226 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065d245-bfe3-48c1-972a-82476efe0be6 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:50.378514 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065d245-bfe3-48c1-972a-82476efe0be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:50.378853 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065d245-bfe3-48c1-972a-82476efe0be6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:50.380076 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d065d245-bfe3-48c1-972a-82476efe0be6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:50.382043 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d065d245-bfe3-48c1-972a-82476efe0be6 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:50.392770 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e1165-b4cd-4069-bcc9-d51344497fc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:50.394725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9e1165-b4cd-4069-bcc9-d51344497fc3 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:50.408407 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1130/2235] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:05:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:05:50.415065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1106/2236] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:05:50 2022] POST /identity/v3/projects => generated 379 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:05:50.416213 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e1165-b4cd-4069-bcc9-d51344497fc3 service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:50.426976 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9e1165-b4cd-4069-bcc9-d51344497fc3 service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:50.502873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f02d66-b0f6-4654-bef2-56a3234698f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:50.503566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f02d66-b0f6-4654-bef2-56a3234698f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:50.503861 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f02d66-b0f6-4654-bef2-56a3234698f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:50.504178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f02d66-b0f6-4654-bef2-56a3234698f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:50.504880 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f02d66-b0f6-4654-bef2-56a3234698f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:50.506823 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f02d66-b0f6-4654-bef2-56a3234698f7 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:50.514006 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:05:50.529042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1131/2237] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:05:50 2022] POST /identity/v3/users => generated 296 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:05:50.533037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065d245-bfe3-48c1-972a-82476efe0be6 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:50.545159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d065d245-bfe3-48c1-972a-82476efe0be6 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:50.549971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d76e5-106d-4867-a666-306d3da1e391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:50.550709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d76e5-106d-4867-a666-306d3da1e391 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:50.551541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d76e5-106d-4867-a666-306d3da1e391 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:50.551826 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4d76e5-106d-4867-a666-306d3da1e391 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:50.557182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d76e5-106d-4867-a666-306d3da1e391 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:50.558933 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4d76e5-106d-4867-a666-306d3da1e391 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:50.576672 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1107/2238] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:05:50.577960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f02d66-b0f6-4654-bef2-56a3234698f7 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:50.589276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f02d66-b0f6-4654-bef2-56a3234698f7 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:50.594670 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edd31a-70de-4e0a-afdd-e253595d6f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:50.595500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edd31a-70de-4e0a-afdd-e253595d6f67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:50.595786 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edd31a-70de-4e0a-afdd-e253595d6f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:50.596067 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05edd31a-70de-4e0a-afdd-e253595d6f67 admin admin] PATH_INFO: `/v3/projects/3611b63c998b4abebe0a7deb01d1f0d5/users/0dadd97b13ce42d8b0954a401cdd1b67/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:50.612784 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edd31a-70de-4e0a-afdd-e253595d6f67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3611b63c998b4abebe0a7deb01d1f0d5, user_id=0dadd97b13ce42d8b0954a401cdd1b67, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:50.619783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05edd31a-70de-4e0a-afdd-e253595d6f67 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:50.638448 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1132/2239] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:05:50 2022] PUT /identity/v3/projects/3611b63c998b4abebe0a7deb01d1f0d5/users/0dadd97b13ce42d8b0954a401cdd1b67/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:05:50.642376 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d76e5-106d-4867-a666-306d3da1e391 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:50.657557 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4d76e5-106d-4867-a666-306d3da1e391 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:50.728211 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac0d54-0f22-476c-b521-e8788f94130b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:50.728892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac0d54-0f22-476c-b521-e8788f94130b admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:50.729185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac0d54-0f22-476c-b521-e8788f94130b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:50.729466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac0d54-0f22-476c-b521-e8788f94130b admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:50.730197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac0d54-0f22-476c-b521-e8788f94130b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:50.731849 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac0d54-0f22-476c-b521-e8788f94130b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:50.743354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1108/2240] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:05:50 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:05:50.747552 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edd31a-70de-4e0a-afdd-e253595d6f67 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:50.758745 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05edd31a-70de-4e0a-afdd-e253595d6f67 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:50.763628 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d24e1-252c-47cf-9393-55f03f4354b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:50.764528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d24e1-252c-47cf-9393-55f03f4354b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:50.764812 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d24e1-252c-47cf-9393-55f03f4354b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:50.765125 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4d24e1-252c-47cf-9393-55f03f4354b2 admin admin] PATH_INFO: `/v3/projects/3611b63c998b4abebe0a7deb01d1f0d5/users/0dadd97b13ce42d8b0954a401cdd1b67/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:50.767477 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d24e1-252c-47cf-9393-55f03f4354b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3611b63c998b4abebe0a7deb01d1f0d5, user_id=0dadd97b13ce42d8b0954a401cdd1b67, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:50.770161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4d24e1-252c-47cf-9393-55f03f4354b2 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:50.783480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1133/2241] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:05:50 2022] PUT /identity/v3/projects/3611b63c998b4abebe0a7deb01d1f0d5/users/0dadd97b13ce42d8b0954a401cdd1b67/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:05:54.823388 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac0d54-0f22-476c-b521-e8788f94130b admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:54.998155 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac0d54-0f22-476c-b521-e8788f94130b admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:55.042761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d24e1-252c-47cf-9393-55f03f4354b2 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:55.045447 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7f4d24e1-252c-47cf-9393-55f03f4354b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:55.047819 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:05:55.094519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55183de-ce01-4f89-a742-2458aaca0734 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:55.095227 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55183de-ce01-4f89-a742-2458aaca0734 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:55.095517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55183de-ce01-4f89-a742-2458aaca0734 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:55.095925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55183de-ce01-4f89-a742-2458aaca0734 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:55.097861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d55183de-ce01-4f89-a742-2458aaca0734 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:55.109211 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55183de-ce01-4f89-a742-2458aaca0734 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:55.117490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55183de-ce01-4f89-a742-2458aaca0734 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:55.128206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1109/2242] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:05:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:05:55.150649 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4d24e1-252c-47cf-9393-55f03f4354b2 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:55.164883 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4553f9-8524-4a14-8ef6-69378a2dc804 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:55.165883 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4553f9-8524-4a14-8ef6-69378a2dc804 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:55.166393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4553f9-8524-4a14-8ef6-69378a2dc804 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:55.166855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4553f9-8524-4a14-8ef6-69378a2dc804 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:55.177666 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4553f9-8524-4a14-8ef6-69378a2dc804 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:55.180002 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4553f9-8524-4a14-8ef6-69378a2dc804 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:55.191261 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1134/2243] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:05:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:05:57.746585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94679176-2d02-483b-ac2b-53d73ffb1db5 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:57.747153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94679176-2d02-483b-ac2b-53d73ffb1db5 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:57.747153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94679176-2d02-483b-ac2b-53d73ffb1db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:57.788490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-94679176-2d02-483b-ac2b-53d73ffb1db5 None None] MFA Rules not processed for user `0dadd97b13ce42d8b0954a401cdd1b67`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:05:57.789921 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-94679176-2d02-483b-ac2b-53d73ffb1db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:57.806743 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4553f9-8524-4a14-8ef6-69378a2dc804 service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:57.808670 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-fa4553f9-8524-4a14-8ef6-69378a2dc804 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:57.887996 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1110/2244] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:05:57 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:05:57.895203 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94679176-2d02-483b-ac2b-53d73ffb1db5 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:57.911447 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4553f9-8524-4a14-8ef6-69378a2dc804 service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:57.921600 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877448f-c37a-4a1c-9da4-fb24c3a9afcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:57.924277 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877448f-c37a-4a1c-9da4-fb24c3a9afcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:57.924515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877448f-c37a-4a1c-9da4-fb24c3a9afcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:57.924777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b877448f-c37a-4a1c-9da4-fb24c3a9afcb admin admin] PATH_INFO: `/v3/users/243e592fbf1844d89f55dd5035a684f3` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:57.927645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b877448f-c37a-4a1c-9da4-fb24c3a9afcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=243e592fbf1844d89f55dd5035a684f3)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:57.931806 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b877448f-c37a-4a1c-9da4-fb24c3a9afcb admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:58.021826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-b877448f-c37a-4a1c-9da4-fb24c3a9afcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '243e592fbf1844d89f55dd5035a684f3'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:58.030164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94679176-2d02-483b-ac2b-53d73ffb1db5 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:58.033658 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-b877448f-c37a-4a1c-9da4-fb24c3a9afcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '243e592fbf1844d89f55dd5035a684f3'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:58.035291 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-b877448f-c37a-4a1c-9da4-fb24c3a9afcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '243e592fbf1844d89f55dd5035a684f3'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:58.036969 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:05:58.040333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68136c2b-ad8d-4cd6-9d26-e67008e63c2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:58.041880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68136c2b-ad8d-4cd6-9d26-e67008e63c2c service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:58.042269 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68136c2b-ad8d-4cd6-9d26-e67008e63c2c service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:58.042714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68136c2b-ad8d-4cd6-9d26-e67008e63c2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:58.056696 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-b877448f-c37a-4a1c-9da4-fb24c3a9afcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '243e592fbf1844d89f55dd5035a684f3'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:58.057743 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-b877448f-c37a-4a1c-9da4-fb24c3a9afcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '243e592fbf1844d89f55dd5035a684f3'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:58.077577 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1135/2245] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:05:57 2022] DELETE /identity/v3/users/243e592fbf1844d89f55dd5035a684f3 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:05:58.119516 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:05:58.136921 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68136c2b-ad8d-4cd6-9d26-e67008e63c2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:58.138579 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68136c2b-ad8d-4cd6-9d26-e67008e63c2c service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:58.150901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1111/2246] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:05:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 256 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:05:58.306948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877448f-c37a-4a1c-9da4-fb24c3a9afcb admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:58.308446 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b877448f-c37a-4a1c-9da4-fb24c3a9afcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:58.402126 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b877448f-c37a-4a1c-9da4-fb24c3a9afcb admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:58.415426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf127955-ace4-4956-88ea-cb90eef2155d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:58.416585 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf127955-ace4-4956-88ea-cb90eef2155d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:58.416921 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf127955-ace4-4956-88ea-cb90eef2155d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:58.417203 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf127955-ace4-4956-88ea-cb90eef2155d admin admin] PATH_INFO: `/v3/projects/60447df5461141e883d207e028b74bc7` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:58.418487 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf127955-ace4-4956-88ea-cb90eef2155d admin admin] RBAC: Authorizing `identity:delete_project(project_id=60447df5461141e883d207e028b74bc7)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:58.420691 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf127955-ace4-4956-88ea-cb90eef2155d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:58.478632 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bf127955-ace4-4956-88ea-cb90eef2155d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60447df5461141e883d207e028b74bc7', 'request_id': 'req-bf127955-ace4-4956-88ea-cb90eef2155d'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:58.485489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bf127955-ace4-4956-88ea-cb90eef2155d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60447df5461141e883d207e028b74bc7', 'request_id': 'req-bf127955-ace4-4956-88ea-cb90eef2155d'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:58.493424 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bf127955-ace4-4956-88ea-cb90eef2155d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60447df5461141e883d207e028b74bc7', 'request_id': 'req-bf127955-ace4-4956-88ea-cb90eef2155d'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:58.511277 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bf127955-ace4-4956-88ea-cb90eef2155d admin admin] The token cache is being invalidate because project 60447df5461141e883d207e028b74bc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:05:58.511673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bf127955-ace4-4956-88ea-cb90eef2155d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:58.512640 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-bf127955-ace4-4956-88ea-cb90eef2155d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:05:58.517647 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1136/2247] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:05:58 2022] DELETE /identity/v3/projects/60447df5461141e883d207e028b74bc7 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:05:58.519498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706fa08-4dbb-4d21-9194-8728bb28a037 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:58.519800 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706fa08-4dbb-4d21-9194-8728bb28a037 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:58.520108 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0706fa08-4dbb-4d21-9194-8728bb28a037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:58.555495 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-0706fa08-4dbb-4d21-9194-8728bb28a037 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:05:58.556665 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-0706fa08-4dbb-4d21-9194-8728bb28a037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:58.668438 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1112/2248] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:05:58.670671 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc698b-dcd4-456e-862d-302e3ba2309b None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:58.670968 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc698b-dcd4-456e-862d-302e3ba2309b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:58.671249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4dc698b-dcd4-456e-862d-302e3ba2309b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:58.700404 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-b4dc698b-dcd4-456e-862d-302e3ba2309b None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:05:58.701779 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b4dc698b-dcd4-456e-862d-302e3ba2309b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:58.719392 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1137/2249] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:05:58.724288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706fa08-4dbb-4d21-9194-8728bb28a037 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:58.734664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0706fa08-4dbb-4d21-9194-8728bb28a037 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:58.754171 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b90a6-e567-4105-830b-1efea31074ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:58.755031 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b90a6-e567-4105-830b-1efea31074ab admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:58.755320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b90a6-e567-4105-830b-1efea31074ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:58.755601 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645b90a6-e567-4105-830b-1efea31074ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:58.756341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645b90a6-e567-4105-830b-1efea31074ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:05:58.756717 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645b90a6-e567-4105-830b-1efea31074ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:58.758835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645b90a6-e567-4105-830b-1efea31074ab admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:58.780089 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1113/2250] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:05:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:05:58.787196 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc698b-dcd4-456e-862d-302e3ba2309b None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:58.802559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4dc698b-dcd4-456e-862d-302e3ba2309b None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:58.809819 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1f8a4-0db3-41fb-8648-5bb862653a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:58.810438 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1f8a4-0db3-41fb-8648-5bb862653a75 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:58.810787 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1f8a4-0db3-41fb-8648-5bb862653a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:58.811089 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf1f8a4-0db3-41fb-8648-5bb862653a75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:58.813634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1f8a4-0db3-41fb-8648-5bb862653a75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:58.815814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf1f8a4-0db3-41fb-8648-5bb862653a75 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:58.851762 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1138/2251] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:05:58 2022] POST /identity/v3/projects => generated 387 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:05:58.856105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b90a6-e567-4105-830b-1efea31074ab admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:58.866172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645b90a6-e567-4105-830b-1efea31074ab admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:58.954799 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b4480-6462-44f4-a9b4-ba1694920c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:58.955976 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b4480-6462-44f4-a9b4-ba1694920c11 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:58.956955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b4480-6462-44f4-a9b4-ba1694920c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:58.957385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b4480-6462-44f4-a9b4-ba1694920c11 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:58.958518 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b4480-6462-44f4-a9b4-ba1694920c11 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:58.960727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b4480-6462-44f4-a9b4-ba1694920c11 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:58.986279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1114/2252] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:05:58 2022] POST /identity/v3/users => generated 294 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:05:58.990657 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1f8a4-0db3-41fb-8648-5bb862653a75 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:59.006845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf1f8a4-0db3-41fb-8648-5bb862653a75 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:59.012116 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e010e-634e-4d54-a08b-c7227036566f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:59.012818 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e010e-634e-4d54-a08b-c7227036566f admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:59.013108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e010e-634e-4d54-a08b-c7227036566f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:59.013389 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58e010e-634e-4d54-a08b-c7227036566f admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:59.014134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e010e-634e-4d54-a08b-c7227036566f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:59.015825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58e010e-634e-4d54-a08b-c7227036566f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:59.032329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1139/2253] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:05:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:05:59.033861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b4480-6462-44f4-a9b4-ba1694920c11 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:59.054631 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b4480-6462-44f4-a9b4-ba1694920c11 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:59.061462 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e49524-9c21-4539-8c21-6081a9a43033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:59.062147 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e49524-9c21-4539-8c21-6081a9a43033 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:59.062524 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e49524-9c21-4539-8c21-6081a9a43033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:59.062917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e49524-9c21-4539-8c21-6081a9a43033 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:59.063729 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e49524-9c21-4539-8c21-6081a9a43033 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:59.065434 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e49524-9c21-4539-8c21-6081a9a43033 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:59.080123 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1115/2254] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:05:59.085854 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e010e-634e-4d54-a08b-c7227036566f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:59.096510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58e010e-634e-4d54-a08b-c7227036566f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:59.101951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ab8ed-65cf-495f-b5ff-2841e78afb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:59.102952 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ab8ed-65cf-495f-b5ff-2841e78afb44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:59.103239 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ab8ed-65cf-495f-b5ff-2841e78afb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:59.103515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556ab8ed-65cf-495f-b5ff-2841e78afb44 admin admin] PATH_INFO: `/v3/projects/94fd377026914b9fa710f04531937501/users/3200dcd23da34711977b18e1c968c627/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:59.122859 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ab8ed-65cf-495f-b5ff-2841e78afb44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94fd377026914b9fa710f04531937501, user_id=3200dcd23da34711977b18e1c968c627, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:59.125723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556ab8ed-65cf-495f-b5ff-2841e78afb44 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:59.139798 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:05:59.150335 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1140/2255] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:05:59 2022] PUT /identity/v3/projects/94fd377026914b9fa710f04531937501/users/3200dcd23da34711977b18e1c968c627/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:05:59.160012 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445979d7-305f-448c-a724-830cb0f3d6ae None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:59.160377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445979d7-305f-448c-a724-830cb0f3d6ae None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:59.160575 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445979d7-305f-448c-a724-830cb0f3d6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:59.232382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-445979d7-305f-448c-a724-830cb0f3d6ae None None] MFA Rules not processed for user `3200dcd23da34711977b18e1c968c627`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:05:59.233653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-445979d7-305f-448c-a724-830cb0f3d6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:59.340043 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:05:59.384348 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1116/2256] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:05:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:05:59.385620 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ab8ed-65cf-495f-b5ff-2841e78afb44 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:59.474233 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556ab8ed-65cf-495f-b5ff-2841e78afb44 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:59.479097 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68b944-e141-4dd2-a925-3fb97e65b335 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:59.479770 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68b944-e141-4dd2-a925-3fb97e65b335 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:59.480056 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68b944-e141-4dd2-a925-3fb97e65b335 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:59.480329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb68b944-e141-4dd2-a925-3fb97e65b335 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:59.481026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb68b944-e141-4dd2-a925-3fb97e65b335 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:59.483458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb68b944-e141-4dd2-a925-3fb97e65b335 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:59.517566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1141/2257] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:05:59 2022] POST /identity/v3/projects => generated 385 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:05:59.519425 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445979d7-305f-448c-a724-830cb0f3d6ae None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:59.531145 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445979d7-305f-448c-a724-830cb0f3d6ae None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:59.593021 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68b944-e141-4dd2-a925-3fb97e65b335 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:59.595131 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-fb68b944-e141-4dd2-a925-3fb97e65b335 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:59.623803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6c6c3-55c1-42be-b4c8-88d35323cb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:59.624671 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6c6c3-55c1-42be-b4c8-88d35323cb7d admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:59.625299 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6c6c3-55c1-42be-b4c8-88d35323cb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:59.625299 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e6c6c3-55c1-42be-b4c8-88d35323cb7d admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:59.626238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e6c6c3-55c1-42be-b4c8-88d35323cb7d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:59.629068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e6c6c3-55c1-42be-b4c8-88d35323cb7d admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:59.655972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1117/2258] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:05:59 2022] POST /identity/v3/users => generated 299 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:05:59.657655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6c6c3-55c1-42be-b4c8-88d35323cb7d admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:59.671674 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e6c6c3-55c1-42be-b4c8-88d35323cb7d admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:59.677584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9378d2-f390-4a3c-83c9-fa3ded8831c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:59.678484 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9378d2-f390-4a3c-83c9-fa3ded8831c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:59.678962 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9378d2-f390-4a3c-83c9-fa3ded8831c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:59.679555 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9378d2-f390-4a3c-83c9-fa3ded8831c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:59.681643 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9378d2-f390-4a3c-83c9-fa3ded8831c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:59.683776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d9378d2-f390-4a3c-83c9-fa3ded8831c2 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:59.700238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1118/2259] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:05:59.704499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9378d2-f390-4a3c-83c9-fa3ded8831c2 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:59.719594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d9378d2-f390-4a3c-83c9-fa3ded8831c2 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:59.728846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d331c6f-8ca3-4bf2-8d19-921e1ec1237d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:59.729789 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d331c6f-8ca3-4bf2-8d19-921e1ec1237d admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:59.730079 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d331c6f-8ca3-4bf2-8d19-921e1ec1237d admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:59.730763 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d331c6f-8ca3-4bf2-8d19-921e1ec1237d admin admin] PATH_INFO: `/v3/projects/e35d02aa1dd34d15ba747dc2bb213d70/users/b1a0b29ed49a43549d16d05a65891f9f/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:59.754979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d331c6f-8ca3-4bf2-8d19-921e1ec1237d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e35d02aa1dd34d15ba747dc2bb213d70, user_id=b1a0b29ed49a43549d16d05a65891f9f, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:59.756835 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb68b944-e141-4dd2-a925-3fb97e65b335 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:59.757669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d331c6f-8ca3-4bf2-8d19-921e1ec1237d admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:59.764332 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cd16d-7a4f-495a-b360-f8ecc50802b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:59.765075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cd16d-7a4f-495a-b360-f8ecc50802b1 service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:59.765513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cd16d-7a4f-495a-b360-f8ecc50802b1 service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:59.765780 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cd16d-7a4f-495a-b360-f8ecc50802b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:59.767843 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ca8cd16d-7a4f-495a-b360-f8ecc50802b1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:59.772721 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1119/2260] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:05:59 2022] PUT /identity/v3/projects/e35d02aa1dd34d15ba747dc2bb213d70/users/b1a0b29ed49a43549d16d05a65891f9f/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:05:59.776986 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d331c6f-8ca3-4bf2-8d19-921e1ec1237d admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:59.843945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8cd16d-7a4f-495a-b360-f8ecc50802b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:59.845721 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8cd16d-7a4f-495a-b360-f8ecc50802b1 service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:59.855052 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d331c6f-8ca3-4bf2-8d19-921e1ec1237d admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:59.857086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1142/2261] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:05:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:05:59.860295 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a445b55a-9c9c-4590-aaad-37951e16288d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:59.861059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445b55a-9c9c-4590-aaad-37951e16288d admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:59.861413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445b55a-9c9c-4590-aaad-37951e16288d admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:59.861772 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a445b55a-9c9c-4590-aaad-37951e16288d admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:59.863245 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445b55a-9c9c-4590-aaad-37951e16288d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:59.865087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a445b55a-9c9c-4590-aaad-37951e16288d admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:59.878710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1120/2262] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:05:59 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:05:59.882728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cd16d-7a4f-495a-b360-f8ecc50802b1 service nova] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:05:59.893897 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cd16d-7a4f-495a-b360-f8ecc50802b1 service nova] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:05:59.898658 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2a6fb-9044-4ec0-b43e-ddebd367a874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:05:59.899558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2a6fb-9044-4ec0-b43e-ddebd367a874 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:59.899842 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2a6fb-9044-4ec0-b43e-ddebd367a874 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:59.900224 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf2a6fb-9044-4ec0-b43e-ddebd367a874 admin admin] PATH_INFO: `/v3/projects/e35d02aa1dd34d15ba747dc2bb213d70/users/b1a0b29ed49a43549d16d05a65891f9f/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:59.902205 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2a6fb-9044-4ec0-b43e-ddebd367a874 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e35d02aa1dd34d15ba747dc2bb213d70, user_id=b1a0b29ed49a43549d16d05a65891f9f, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:05:59.905033 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf2a6fb-9044-4ec0-b43e-ddebd367a874 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:05:59.920202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1143/2263] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:05:59 2022] PUT /identity/v3/projects/e35d02aa1dd34d15ba747dc2bb213d70/users/b1a0b29ed49a43549d16d05a65891f9f/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:05:59.932078 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fb396-38c3-49fd-b761-59562c4ae03f None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:05:59.932369 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fb396-38c3-49fd-b761-59562c4ae03f None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:05:59.932650 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569fb396-38c3-49fd-b761-59562c4ae03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:05:59.965396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-569fb396-38c3-49fd-b761-59562c4ae03f None None] MFA Rules not processed for user `b1a0b29ed49a43549d16d05a65891f9f`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:05:59.966800 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-569fb396-38c3-49fd-b761-59562c4ae03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:05:59.992911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:00.051439 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1121/2264] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:05:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:00.148227 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2a6fb-9044-4ec0-b43e-ddebd367a874 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:00.155068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-fdf2a6fb-9044-4ec0-b43e-ddebd367a874 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:00.215117 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:00.306288 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf2a6fb-9044-4ec0-b43e-ddebd367a874 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:00.310195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53224817-3d02-4316-9b87-e12fd2b8e7d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:00.311217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53224817-3d02-4316-9b87-e12fd2b8e7d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:00.311335 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53224817-3d02-4316-9b87-e12fd2b8e7d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:00.311591 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53224817-3d02-4316-9b87-e12fd2b8e7d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:00.323073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53224817-3d02-4316-9b87-e12fd2b8e7d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:00.325316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53224817-3d02-4316-9b87-e12fd2b8e7d2 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:00.338395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1144/2265] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:00 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:00.376592 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fb396-38c3-49fd-b761-59562c4ae03f None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:00.447434 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569fb396-38c3-49fd-b761-59562c4ae03f None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:00.453353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6abd1f-9e20-45fe-89e8-197f70f6ca55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:00.454819 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6abd1f-9e20-45fe-89e8-197f70f6ca55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:00.454819 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6abd1f-9e20-45fe-89e8-197f70f6ca55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:00.455096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6abd1f-9e20-45fe-89e8-197f70f6ca55 admin admin] PATH_INFO: `/v3/users/3200dcd23da34711977b18e1c968c627` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:00.456461 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6abd1f-9e20-45fe-89e8-197f70f6ca55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3200dcd23da34711977b18e1c968c627)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:00.458780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6abd1f-9e20-45fe-89e8-197f70f6ca55 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:00.518954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1d6abd1f-9e20-45fe-89e8-197f70f6ca55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3200dcd23da34711977b18e1c968c627'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:00.519661 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1d6abd1f-9e20-45fe-89e8-197f70f6ca55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3200dcd23da34711977b18e1c968c627'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:00.528819 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1d6abd1f-9e20-45fe-89e8-197f70f6ca55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3200dcd23da34711977b18e1c968c627'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:00.529689 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1d6abd1f-9e20-45fe-89e8-197f70f6ca55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3200dcd23da34711977b18e1c968c627'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:00.543195 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1d6abd1f-9e20-45fe-89e8-197f70f6ca55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3200dcd23da34711977b18e1c968c627'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:00.563312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1122/2266] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:06:00 2022] DELETE /identity/v3/users/3200dcd23da34711977b18e1c968c627 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:00.568815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3dab8c-f532-48d1-95ae-f97edfb5cb23 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:00.569466 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3dab8c-f532-48d1-95ae-f97edfb5cb23 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:00.569795 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3dab8c-f532-48d1-95ae-f97edfb5cb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:00.595218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-ba3dab8c-f532-48d1-95ae-f97edfb5cb23 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:00.597225 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ba3dab8c-f532-48d1-95ae-f97edfb5cb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:00.666678 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1145/2267] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:00.678506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6abd1f-9e20-45fe-89e8-197f70f6ca55 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:00.680020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-1d6abd1f-9e20-45fe-89e8-197f70f6ca55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:00.725537 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:00.750955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6abd1f-9e20-45fe-89e8-197f70f6ca55 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:00.755785 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ac883-caab-4c0d-8270-bbccbb4bb6b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:00.756584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ac883-caab-4c0d-8270-bbccbb4bb6b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:00.756882 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ac883-caab-4c0d-8270-bbccbb4bb6b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:00.757726 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6ac883-caab-4c0d-8270-bbccbb4bb6b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:00.769050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ac883-caab-4c0d-8270-bbccbb4bb6b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:00.770519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6ac883-caab-4c0d-8270-bbccbb4bb6b4 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:00.785271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1123/2268] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:01.086535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3dab8c-f532-48d1-95ae-f97edfb5cb23 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:01.088005 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ba3dab8c-f532-48d1-95ae-f97edfb5cb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:01.102813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3dab8c-f532-48d1-95ae-f97edfb5cb23 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:01.107575 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa71c3-72b9-481b-8367-b02cddcc9d5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:01.108458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa71c3-72b9-481b-8367-b02cddcc9d5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:01.108729 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa71c3-72b9-481b-8367-b02cddcc9d5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:01.109003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fa71c3-72b9-481b-8367-b02cddcc9d5e admin admin] PATH_INFO: `/v3/projects/94fd377026914b9fa710f04531937501` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:01.111283 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa71c3-72b9-481b-8367-b02cddcc9d5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=94fd377026914b9fa710f04531937501)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:01.113716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fa71c3-72b9-481b-8367-b02cddcc9d5e admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:01.167810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-35fa71c3-72b9-481b-8367-b02cddcc9d5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94fd377026914b9fa710f04531937501', 'request_id': 'req-35fa71c3-72b9-481b-8367-b02cddcc9d5e'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:01.174400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-35fa71c3-72b9-481b-8367-b02cddcc9d5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94fd377026914b9fa710f04531937501', 'request_id': 'req-35fa71c3-72b9-481b-8367-b02cddcc9d5e'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:01.180604 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-35fa71c3-72b9-481b-8367-b02cddcc9d5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94fd377026914b9fa710f04531937501', 'request_id': 'req-35fa71c3-72b9-481b-8367-b02cddcc9d5e'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:01.193613 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-35fa71c3-72b9-481b-8367-b02cddcc9d5e admin admin] The token cache is being invalidate because project 94fd377026914b9fa710f04531937501 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:06:01.193903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-35fa71c3-72b9-481b-8367-b02cddcc9d5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:01.194568 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-35fa71c3-72b9-481b-8367-b02cddcc9d5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:01.196142 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1146/2269] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:06:01 2022] DELETE /identity/v3/projects/94fd377026914b9fa710f04531937501 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:01.200261 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ac883-caab-4c0d-8270-bbccbb4bb6b4 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:01.214324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6ac883-caab-4c0d-8270-bbccbb4bb6b4 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:01.273249 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73ef8be-f774-4746-8952-865fc263d82e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:01.274143 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ef8be-f774-4746-8952-865fc263d82e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:01.274433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ef8be-f774-4746-8952-865fc263d82e admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:01.274776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73ef8be-f774-4746-8952-865fc263d82e admin admin] PATH_INFO: `/v3/users/b1a0b29ed49a43549d16d05a65891f9f` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:01.275984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ef8be-f774-4746-8952-865fc263d82e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1a0b29ed49a43549d16d05a65891f9f)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:01.277938 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73ef8be-f774-4746-8952-865fc263d82e admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:01.335280 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d73ef8be-f774-4746-8952-865fc263d82e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1a0b29ed49a43549d16d05a65891f9f'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:01.337013 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d73ef8be-f774-4746-8952-865fc263d82e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1a0b29ed49a43549d16d05a65891f9f'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:01.361892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d73ef8be-f774-4746-8952-865fc263d82e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1a0b29ed49a43549d16d05a65891f9f'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:01.365793 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d73ef8be-f774-4746-8952-865fc263d82e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1a0b29ed49a43549d16d05a65891f9f'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:01.394831 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d73ef8be-f774-4746-8952-865fc263d82e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1a0b29ed49a43549d16d05a65891f9f'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:01.439946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1124/2270] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:06:01 2022] DELETE /identity/v3/users/b1a0b29ed49a43549d16d05a65891f9f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:02.075036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa71c3-72b9-481b-8367-b02cddcc9d5e admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:02.077007 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-35fa71c3-72b9-481b-8367-b02cddcc9d5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:02.098428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fa71c3-72b9-481b-8367-b02cddcc9d5e admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:02.148318 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:02.182297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d2b878-fed6-4e55-8a38-391609f7cbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:02.183173 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2b878-fed6-4e55-8a38-391609f7cbe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:02.183669 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2b878-fed6-4e55-8a38-391609f7cbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:02.183981 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d2b878-fed6-4e55-8a38-391609f7cbe2 admin admin] PATH_INFO: `/v3/projects/e35d02aa1dd34d15ba747dc2bb213d70` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:02.185663 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2b878-fed6-4e55-8a38-391609f7cbe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e35d02aa1dd34d15ba747dc2bb213d70)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:02.189314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d2b878-fed6-4e55-8a38-391609f7cbe2 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:02.253785 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-77d2b878-fed6-4e55-8a38-391609f7cbe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e35d02aa1dd34d15ba747dc2bb213d70', 'request_id': 'req-77d2b878-fed6-4e55-8a38-391609f7cbe2'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:02.261142 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-77d2b878-fed6-4e55-8a38-391609f7cbe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e35d02aa1dd34d15ba747dc2bb213d70', 'request_id': 'req-77d2b878-fed6-4e55-8a38-391609f7cbe2'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:02.269133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-77d2b878-fed6-4e55-8a38-391609f7cbe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e35d02aa1dd34d15ba747dc2bb213d70', 'request_id': 'req-77d2b878-fed6-4e55-8a38-391609f7cbe2'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:02.287264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-77d2b878-fed6-4e55-8a38-391609f7cbe2 admin admin] The token cache is being invalidate because project e35d02aa1dd34d15ba747dc2bb213d70 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:06:02.287611 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-77d2b878-fed6-4e55-8a38-391609f7cbe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:02.288377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-77d2b878-fed6-4e55-8a38-391609f7cbe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:02.289971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1147/2271] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:06:02 2022] DELETE /identity/v3/projects/e35d02aa1dd34d15ba747dc2bb213d70 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:02.295108 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815229e6-7acc-4663-b9ea-042035f44091 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:02.296031 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815229e6-7acc-4663-b9ea-042035f44091 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:02.296501 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815229e6-7acc-4663-b9ea-042035f44091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:02.316788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-815229e6-7acc-4663-b9ea-042035f44091 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:02.318020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-815229e6-7acc-4663-b9ea-042035f44091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:02.377346 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:02.388954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1125/2272] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:06:02.396733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39009953-6c54-4944-b0c3-f4e000aa294b None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:02.396966 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39009953-6c54-4944-b0c3-f4e000aa294b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:02.397244 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39009953-6c54-4944-b0c3-f4e000aa294b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:02.416792 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-39009953-6c54-4944-b0c3-f4e000aa294b None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:02.418028 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-39009953-6c54-4944-b0c3-f4e000aa294b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:02.428018 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1148/2273] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:02.432392 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815229e6-7acc-4663-b9ea-042035f44091 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:02.442911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815229e6-7acc-4663-b9ea-042035f44091 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:02.447928 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83354c9d-a837-4dc9-8dbe-a69fa7e2344a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:02.448602 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83354c9d-a837-4dc9-8dbe-a69fa7e2344a admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:02.448890 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83354c9d-a837-4dc9-8dbe-a69fa7e2344a admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:02.449167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83354c9d-a837-4dc9-8dbe-a69fa7e2344a admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:02.449927 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83354c9d-a837-4dc9-8dbe-a69fa7e2344a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:06:02.450238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83354c9d-a837-4dc9-8dbe-a69fa7e2344a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:02.451903 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83354c9d-a837-4dc9-8dbe-a69fa7e2344a admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:02.467111 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1126/2274] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:06:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:06:02.478930 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39009953-6c54-4944-b0c3-f4e000aa294b None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:02.488746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39009953-6c54-4944-b0c3-f4e000aa294b None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:02.493453 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9913e-d6fd-4cf8-a862-435c3165e42b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:02.494115 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9913e-d6fd-4cf8-a862-435c3165e42b admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:02.494402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9913e-d6fd-4cf8-a862-435c3165e42b admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:02.494721 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9913e-d6fd-4cf8-a862-435c3165e42b admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:02.495412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9913e-d6fd-4cf8-a862-435c3165e42b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:02.497298 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9913e-d6fd-4cf8-a862-435c3165e42b admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:02.524888 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1149/2275] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:06:02 2022] POST /identity/v3/projects => generated 379 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:06:02.528899 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83354c9d-a837-4dc9-8dbe-a69fa7e2344a admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:02.540184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83354c9d-a837-4dc9-8dbe-a69fa7e2344a admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:02.598108 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376789f-68fe-4856-8fda-a945d7844fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:02.598825 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376789f-68fe-4856-8fda-a945d7844fdd admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:02.599214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376789f-68fe-4856-8fda-a945d7844fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:02.599527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a376789f-68fe-4856-8fda-a945d7844fdd admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:02.600270 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a376789f-68fe-4856-8fda-a945d7844fdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:02.602374 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a376789f-68fe-4856-8fda-a945d7844fdd admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:02.626075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1127/2276] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:06:02 2022] POST /identity/v3/users => generated 290 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:06:02.627284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9913e-d6fd-4cf8-a862-435c3165e42b admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:02.639463 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9913e-d6fd-4cf8-a862-435c3165e42b admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:02.644705 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f4cff-3cee-462a-8132-64d030e79b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:02.645483 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f4cff-3cee-462a-8132-64d030e79b29 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:02.645842 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f4cff-3cee-462a-8132-64d030e79b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:02.646524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063f4cff-3cee-462a-8132-64d030e79b29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:02.647633 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f4cff-3cee-462a-8132-64d030e79b29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:02.649664 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063f4cff-3cee-462a-8132-64d030e79b29 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:02.673587 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1150/2277] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:06:02.674993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376789f-68fe-4856-8fda-a945d7844fdd admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:02.689092 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a376789f-68fe-4856-8fda-a945d7844fdd admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:02.699311 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410a8bc8-c2e9-45ba-9e80-f61b82838fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:02.699983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a8bc8-c2e9-45ba-9e80-f61b82838fcd admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:02.700274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a8bc8-c2e9-45ba-9e80-f61b82838fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:02.700557 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410a8bc8-c2e9-45ba-9e80-f61b82838fcd admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:02.701310 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a8bc8-c2e9-45ba-9e80-f61b82838fcd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:02.703062 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410a8bc8-c2e9-45ba-9e80-f61b82838fcd admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:02.719880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1128/2278] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:02 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:06:02.723993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f4cff-3cee-462a-8132-64d030e79b29 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:02.739808 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063f4cff-3cee-462a-8132-64d030e79b29 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:02.744849 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bab98-1214-4627-8efb-847934cf9997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:02.745755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bab98-1214-4627-8efb-847934cf9997 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:02.746037 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bab98-1214-4627-8efb-847934cf9997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:02.746310 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722bab98-1214-4627-8efb-847934cf9997 admin admin] PATH_INFO: `/v3/projects/eeaa0b4deee745f0abd3f4b443585dc4/users/9e736b0e02f54511b48156d8c6307890/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:02.765765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bab98-1214-4627-8efb-847934cf9997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eeaa0b4deee745f0abd3f4b443585dc4, user_id=9e736b0e02f54511b48156d8c6307890, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:02.768673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722bab98-1214-4627-8efb-847934cf9997 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:02.785263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1151/2279] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:06:02 2022] PUT /identity/v3/projects/eeaa0b4deee745f0abd3f4b443585dc4/users/9e736b0e02f54511b48156d8c6307890/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:02.800410 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4807fa6-3bf6-4c69-b6df-5abd3f2e628b None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:02.801319 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4807fa6-3bf6-4c69-b6df-5abd3f2e628b None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:02.801848 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4807fa6-3bf6-4c69-b6df-5abd3f2e628b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:02.837732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-b4807fa6-3bf6-4c69-b6df-5abd3f2e628b None None] MFA Rules not processed for user `9e736b0e02f54511b48156d8c6307890`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:02.839146 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-b4807fa6-3bf6-4c69-b6df-5abd3f2e628b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:02.943170 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1129/2280] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:02 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:02.949319 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bab98-1214-4627-8efb-847934cf9997 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:03.053783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722bab98-1214-4627-8efb-847934cf9997 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:03.071129 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659266a0-f0d7-4f8a-9237-9b18afe57fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:03.072938 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659266a0-f0d7-4f8a-9237-9b18afe57fb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:03.073488 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659266a0-f0d7-4f8a-9237-9b18afe57fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:03.073955 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659266a0-f0d7-4f8a-9237-9b18afe57fb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:03.077512 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659266a0-f0d7-4f8a-9237-9b18afe57fb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:03.083528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659266a0-f0d7-4f8a-9237-9b18afe57fb7 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:03.114577 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:03.160832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1152/2281] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:06:02 2022] POST /identity/v3/projects => generated 379 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:06:03.164369 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4807fa6-3bf6-4c69-b6df-5abd3f2e628b None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:03.176662 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4807fa6-3bf6-4c69-b6df-5abd3f2e628b None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:03.255643 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:03.273431 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a93e9-429a-4d4c-81d4-7431b7995e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:03.274135 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a93e9-429a-4d4c-81d4-7431b7995e28 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:03.274894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a93e9-429a-4d4c-81d4-7431b7995e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:03.275183 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443a93e9-429a-4d4c-81d4-7431b7995e28 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:03.276050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a93e9-429a-4d4c-81d4-7431b7995e28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:03.278712 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443a93e9-429a-4d4c-81d4-7431b7995e28 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:03.318474 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1130/2282] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:06:03 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:06:03.322315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659266a0-f0d7-4f8a-9237-9b18afe57fb7 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:03.338819 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659266a0-f0d7-4f8a-9237-9b18afe57fb7 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:03.345739 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5431b2-0319-4a2b-8ede-535c3c941486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:03.346507 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5431b2-0319-4a2b-8ede-535c3c941486 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:03.347128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5431b2-0319-4a2b-8ede-535c3c941486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:03.347128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5431b2-0319-4a2b-8ede-535c3c941486 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:03.348234 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5431b2-0319-4a2b-8ede-535c3c941486 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:03.349944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5431b2-0319-4a2b-8ede-535c3c941486 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:03.372567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1153/2283] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:06:03.376671 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a93e9-429a-4d4c-81d4-7431b7995e28 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:03.388931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443a93e9-429a-4d4c-81d4-7431b7995e28 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:03.393603 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4158feb-eabe-4e5c-b77f-7474078ccef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:03.394287 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4158feb-eabe-4e5c-b77f-7474078ccef1 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:03.394572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4158feb-eabe-4e5c-b77f-7474078ccef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:03.394950 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4158feb-eabe-4e5c-b77f-7474078ccef1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:03.395680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4158feb-eabe-4e5c-b77f-7474078ccef1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:03.397282 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4158feb-eabe-4e5c-b77f-7474078ccef1 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:03.408457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1131/2284] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:06:03.412888 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5431b2-0319-4a2b-8ede-535c3c941486 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:03.431373 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5431b2-0319-4a2b-8ede-535c3c941486 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:03.436230 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc69ce3-a5e7-4514-96b0-4b277c7c4125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:03.437143 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc69ce3-a5e7-4514-96b0-4b277c7c4125 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:03.437430 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc69ce3-a5e7-4514-96b0-4b277c7c4125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:03.437712 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc69ce3-a5e7-4514-96b0-4b277c7c4125 admin admin] PATH_INFO: `/v3/projects/1e7b344d68134e508864a9c6d6dabfb3/users/ce9b795a23c946118213b2c89c0ad72b/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:03.454897 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc69ce3-a5e7-4514-96b0-4b277c7c4125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e7b344d68134e508864a9c6d6dabfb3, user_id=ce9b795a23c946118213b2c89c0ad72b, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:03.457896 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc69ce3-a5e7-4514-96b0-4b277c7c4125 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:03.471194 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1154/2285] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:06:03 2022] PUT /identity/v3/projects/1e7b344d68134e508864a9c6d6dabfb3/users/ce9b795a23c946118213b2c89c0ad72b/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:03.482638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83884e2d-ccef-4d84-8705-61df46cf8348 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:03.483167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83884e2d-ccef-4d84-8705-61df46cf8348 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:03.483504 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83884e2d-ccef-4d84-8705-61df46cf8348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:03.516267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-83884e2d-ccef-4d84-8705-61df46cf8348 None None] MFA Rules not processed for user `ce9b795a23c946118213b2c89c0ad72b`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:03.517575 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-83884e2d-ccef-4d84-8705-61df46cf8348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:03.603765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1132/2286] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:03 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:06:03.608060 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc69ce3-a5e7-4514-96b0-4b277c7c4125 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:03.694914 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc69ce3-a5e7-4514-96b0-4b277c7c4125 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:03.703055 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2bd5e2-4ac7-44bd-b205-ab46f204efa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:03.704037 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bd5e2-4ac7-44bd-b205-ab46f204efa7 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:03.704407 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bd5e2-4ac7-44bd-b205-ab46f204efa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:03.704767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2bd5e2-4ac7-44bd-b205-ab46f204efa7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:03.705623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2bd5e2-4ac7-44bd-b205-ab46f204efa7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:03.708032 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2bd5e2-4ac7-44bd-b205-ab46f204efa7 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:03.752104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1155/2287] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:06:03 2022] POST /identity/v3/projects => generated 379 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:06:03.753860 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83884e2d-ccef-4d84-8705-61df46cf8348 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:03.766005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83884e2d-ccef-4d84-8705-61df46cf8348 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:03.835458 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9030b9-cc37-4d0c-853a-799d89621dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:03.836236 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9030b9-cc37-4d0c-853a-799d89621dc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:03.836430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9030b9-cc37-4d0c-853a-799d89621dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:03.836714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9030b9-cc37-4d0c-853a-799d89621dc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:03.837413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9030b9-cc37-4d0c-853a-799d89621dc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:03.839438 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9030b9-cc37-4d0c-853a-799d89621dc1 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:03.865694 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1133/2288] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:06:03 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:06:03.867081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2bd5e2-4ac7-44bd-b205-ab46f204efa7 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:03.878650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2bd5e2-4ac7-44bd-b205-ab46f204efa7 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:03.883886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f9ed8-b6db-49a7-a1f7-8a12c9fe2608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:03.884575 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f9ed8-b6db-49a7-a1f7-8a12c9fe2608 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:03.885011 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f9ed8-b6db-49a7-a1f7-8a12c9fe2608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:03.885309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2f9ed8-b6db-49a7-a1f7-8a12c9fe2608 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:03.886056 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f9ed8-b6db-49a7-a1f7-8a12c9fe2608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:03.887751 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb2f9ed8-b6db-49a7-a1f7-8a12c9fe2608 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:03.905412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1156/2289] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:06:03.910078 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9030b9-cc37-4d0c-853a-799d89621dc1 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:03.922789 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9030b9-cc37-4d0c-853a-799d89621dc1 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:03.927552 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812a09a-a1ac-4328-a9a0-8916332bbc53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:03.928448 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812a09a-a1ac-4328-a9a0-8916332bbc53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:03.928737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812a09a-a1ac-4328-a9a0-8916332bbc53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:03.929016 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1812a09a-a1ac-4328-a9a0-8916332bbc53 admin admin] PATH_INFO: `/v3/projects/953852ace11742cc96f09875d8aa3901/users/7dd47b2b7dc945418e477c75d640ad39/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:03.947424 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812a09a-a1ac-4328-a9a0-8916332bbc53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=953852ace11742cc96f09875d8aa3901, user_id=7dd47b2b7dc945418e477c75d640ad39, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:03.950171 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1812a09a-a1ac-4328-a9a0-8916332bbc53 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:03.965814 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1134/2290] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:06:03 2022] PUT /identity/v3/projects/953852ace11742cc96f09875d8aa3901/users/7dd47b2b7dc945418e477c75d640ad39/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:03.969780 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f9ed8-b6db-49a7-a1f7-8a12c9fe2608 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:03.981614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2f9ed8-b6db-49a7-a1f7-8a12c9fe2608 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:04.054752 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05745566-687f-4b99-a1b4-db232e36c024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:04.055441 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05745566-687f-4b99-a1b4-db232e36c024 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:04.055733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05745566-687f-4b99-a1b4-db232e36c024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:04.056050 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05745566-687f-4b99-a1b4-db232e36c024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:04.056790 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05745566-687f-4b99-a1b4-db232e36c024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:04.058388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05745566-687f-4b99-a1b4-db232e36c024 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:04.072516 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1157/2291] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:03 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:06:04.073976 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812a09a-a1ac-4328-a9a0-8916332bbc53 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:04.085564 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1812a09a-a1ac-4328-a9a0-8916332bbc53 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:04.090624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62614b74-2f0b-45d5-b76e-b1eae6e56b5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:04.091655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62614b74-2f0b-45d5-b76e-b1eae6e56b5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:04.091976 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62614b74-2f0b-45d5-b76e-b1eae6e56b5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:04.092268 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62614b74-2f0b-45d5-b76e-b1eae6e56b5d admin admin] PATH_INFO: `/v3/projects/953852ace11742cc96f09875d8aa3901/users/7dd47b2b7dc945418e477c75d640ad39/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:04.094648 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62614b74-2f0b-45d5-b76e-b1eae6e56b5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=953852ace11742cc96f09875d8aa3901, user_id=7dd47b2b7dc945418e477c75d640ad39, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:04.097631 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62614b74-2f0b-45d5-b76e-b1eae6e56b5d admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:04.110051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1135/2292] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:06:04 2022] PUT /identity/v3/projects/953852ace11742cc96f09875d8aa3901/users/7dd47b2b7dc945418e477c75d640ad39/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:04.120650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec5a27-9a84-413a-ab0f-ca8ab8a1419e None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:04.121110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec5a27-9a84-413a-ab0f-ca8ab8a1419e None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:04.121205 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec5a27-9a84-413a-ab0f-ca8ab8a1419e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:04.147246 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-66ec5a27-9a84-413a-ab0f-ca8ab8a1419e None None] MFA Rules not processed for user `7dd47b2b7dc945418e477c75d640ad39`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:04.148686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-66ec5a27-9a84-413a-ab0f-ca8ab8a1419e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:04.159311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:04.220832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1158/2293] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:04.231956 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62614b74-2f0b-45d5-b76e-b1eae6e56b5d admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:04.233672 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-62614b74-2f0b-45d5-b76e-b1eae6e56b5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:04.249437 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:04.297490 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62614b74-2f0b-45d5-b76e-b1eae6e56b5d admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:04.301714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee1ecd-92c7-4e50-9213-5a5a3bec5c79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:04.302392 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee1ecd-92c7-4e50-9213-5a5a3bec5c79 service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:04.302707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee1ecd-92c7-4e50-9213-5a5a3bec5c79 service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:04.302989 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ee1ecd-92c7-4e50-9213-5a5a3bec5c79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:04.312340 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee1ecd-92c7-4e50-9213-5a5a3bec5c79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:04.313745 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ee1ecd-92c7-4e50-9213-5a5a3bec5c79 service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:04.323668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1136/2294] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:04.368083 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec5a27-9a84-413a-ab0f-ca8ab8a1419e None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:04.369497 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-66ec5a27-9a84-413a-ab0f-ca8ab8a1419e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:04.381877 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec5a27-9a84-413a-ab0f-ca8ab8a1419e None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:04.385519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3bf4f-8609-42bf-9796-0b5bc9f2b662 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:04.386166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3bf4f-8609-42bf-9796-0b5bc9f2b662 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:04.386762 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3bf4f-8609-42bf-9796-0b5bc9f2b662 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:04.387044 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c3bf4f-8609-42bf-9796-0b5bc9f2b662 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:04.457599 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3bf4f-8609-42bf-9796-0b5bc9f2b662 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:04.459323 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c3bf4f-8609-42bf-9796-0b5bc9f2b662 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:04.477089 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1159/2295] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:04 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:06.615474 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee1ecd-92c7-4e50-9213-5a5a3bec5c79 service cinder] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:06.617859 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-44ee1ecd-92c7-4e50-9213-5a5a3bec5c79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:06.718484 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ee1ecd-92c7-4e50-9213-5a5a3bec5c79 service cinder] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:06.725709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2aec69-7e97-44f0-a38d-fd1274e313e2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:06.726384 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2aec69-7e97-44f0-a38d-fd1274e313e2 service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:06.726703 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2aec69-7e97-44f0-a38d-fd1274e313e2 service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:06.726988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2aec69-7e97-44f0-a38d-fd1274e313e2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:06.728820 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-ba2aec69-7e97-44f0-a38d-fd1274e313e2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:06.799458 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2aec69-7e97-44f0-a38d-fd1274e313e2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:06.801060 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2aec69-7e97-44f0-a38d-fd1274e313e2 service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:06.818409 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1137/2296] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:06 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:07.503150 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3bf4f-8609-42bf-9796-0b5bc9f2b662 service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:07.570366 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c3bf4f-8609-42bf-9796-0b5bc9f2b662 service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:07.574929 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1a51c-9bf7-4880-a064-d856bf246a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:07.575808 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1a51c-9bf7-4880-a064-d856bf246a7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:07.576091 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1a51c-9bf7-4880-a064-d856bf246a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:07.576366 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b1a51c-9bf7-4880-a064-d856bf246a7b admin admin] PATH_INFO: `/v3/users/9e736b0e02f54511b48156d8c6307890` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:07.577552 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1a51c-9bf7-4880-a064-d856bf246a7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e736b0e02f54511b48156d8c6307890)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:07.579553 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b1a51c-9bf7-4880-a064-d856bf246a7b admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:07.628632 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a2b1a51c-9bf7-4880-a064-d856bf246a7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e736b0e02f54511b48156d8c6307890'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:07.629825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:07.636938 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a2b1a51c-9bf7-4880-a064-d856bf246a7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e736b0e02f54511b48156d8c6307890'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:07.637609 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a2b1a51c-9bf7-4880-a064-d856bf246a7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e736b0e02f54511b48156d8c6307890'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:07.649098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a2b1a51c-9bf7-4880-a064-d856bf246a7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e736b0e02f54511b48156d8c6307890'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:07.649724 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a2b1a51c-9bf7-4880-a064-d856bf246a7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e736b0e02f54511b48156d8c6307890'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:07.662860 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1160/2297] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:06:07 2022] DELETE /identity/v3/users/9e736b0e02f54511b48156d8c6307890 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:07.668536 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed7198-4570-4653-8a8d-ab50df4252b8 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:07.669017 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed7198-4570-4653-8a8d-ab50df4252b8 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:07.669172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ed7198-4570-4653-8a8d-ab50df4252b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:07.687194 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-d5ed7198-4570-4653-8a8d-ab50df4252b8 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:07.688387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d5ed7198-4570-4653-8a8d-ab50df4252b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:07.743489 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:07.759525 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1138/2298] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:07.771940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1a51c-9bf7-4880-a064-d856bf246a7b admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:07.774091 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a2b1a51c-9bf7-4880-a064-d856bf246a7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:07.845502 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b1a51c-9bf7-4880-a064-d856bf246a7b admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:07.849326 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeac456-b72c-4b72-9d7f-fb99f00b982a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:07.849968 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeac456-b72c-4b72-9d7f-fb99f00b982a service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:07.850250 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeac456-b72c-4b72-9d7f-fb99f00b982a service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:07.850527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeac456-b72c-4b72-9d7f-fb99f00b982a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:07.859405 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeac456-b72c-4b72-9d7f-fb99f00b982a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:07.861035 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeac456-b72c-4b72-9d7f-fb99f00b982a service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:07.870979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1161/2299] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:06:08.148636 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed7198-4570-4653-8a8d-ab50df4252b8 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:08.150342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d5ed7198-4570-4653-8a8d-ab50df4252b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:08.184113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ed7198-4570-4653-8a8d-ab50df4252b8 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:08.190412 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16863b-536b-4ff7-8a96-0657e62f4cfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:08.191576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16863b-536b-4ff7-8a96-0657e62f4cfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:08.191863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16863b-536b-4ff7-8a96-0657e62f4cfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:08.192144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16863b-536b-4ff7-8a96-0657e62f4cfe admin admin] PATH_INFO: `/v3/projects/eeaa0b4deee745f0abd3f4b443585dc4` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:08.193447 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16863b-536b-4ff7-8a96-0657e62f4cfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=eeaa0b4deee745f0abd3f4b443585dc4)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:08.195386 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16863b-536b-4ff7-8a96-0657e62f4cfe admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:08.286154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-9a16863b-536b-4ff7-8a96-0657e62f4cfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eeaa0b4deee745f0abd3f4b443585dc4', 'request_id': 'req-9a16863b-536b-4ff7-8a96-0657e62f4cfe'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:08.293409 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-9a16863b-536b-4ff7-8a96-0657e62f4cfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eeaa0b4deee745f0abd3f4b443585dc4', 'request_id': 'req-9a16863b-536b-4ff7-8a96-0657e62f4cfe'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:08.299957 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-9a16863b-536b-4ff7-8a96-0657e62f4cfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eeaa0b4deee745f0abd3f4b443585dc4', 'request_id': 'req-9a16863b-536b-4ff7-8a96-0657e62f4cfe'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:08.320967 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-9a16863b-536b-4ff7-8a96-0657e62f4cfe admin admin] The token cache is being invalidate because project eeaa0b4deee745f0abd3f4b443585dc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:06:08.321386 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-9a16863b-536b-4ff7-8a96-0657e62f4cfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:08.322286 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-9a16863b-536b-4ff7-8a96-0657e62f4cfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:08.324409 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1139/2300] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:06:08 2022] DELETE /identity/v3/projects/eeaa0b4deee745f0abd3f4b443585dc4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:08.328509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeac456-b72c-4b72-9d7f-fb99f00b982a service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:08.339268 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeac456-b72c-4b72-9d7f-fb99f00b982a service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:08.402456 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fcbfc-98a8-40a1-8b7b-4031ccd05591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:08.403584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fcbfc-98a8-40a1-8b7b-4031ccd05591 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:08.403876 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fcbfc-98a8-40a1-8b7b-4031ccd05591 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:08.404160 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51fcbfc-98a8-40a1-8b7b-4031ccd05591 admin admin] PATH_INFO: `/v3/users/ce9b795a23c946118213b2c89c0ad72b` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:08.405367 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51fcbfc-98a8-40a1-8b7b-4031ccd05591 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9b795a23c946118213b2c89c0ad72b)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:08.408216 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51fcbfc-98a8-40a1-8b7b-4031ccd05591 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:08.463078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a51fcbfc-98a8-40a1-8b7b-4031ccd05591 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9b795a23c946118213b2c89c0ad72b'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:08.471446 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a51fcbfc-98a8-40a1-8b7b-4031ccd05591 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9b795a23c946118213b2c89c0ad72b'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:08.472176 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a51fcbfc-98a8-40a1-8b7b-4031ccd05591 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9b795a23c946118213b2c89c0ad72b'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:08.486237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:08.488129 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a51fcbfc-98a8-40a1-8b7b-4031ccd05591 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9b795a23c946118213b2c89c0ad72b'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:08.489079 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a51fcbfc-98a8-40a1-8b7b-4031ccd05591 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9b795a23c946118213b2c89c0ad72b'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:08.507387 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1162/2301] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:06:08 2022] DELETE /identity/v3/users/ce9b795a23c946118213b2c89c0ad72b => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:08.834705 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16863b-536b-4ff7-8a96-0657e62f4cfe admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:08.837433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-9a16863b-536b-4ff7-8a96-0657e62f4cfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:08.860624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16863b-536b-4ff7-8a96-0657e62f4cfe admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:08.923825 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c5f9b-b959-477d-9e5c-6b8e7b824a82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:08.925225 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c5f9b-b959-477d-9e5c-6b8e7b824a82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:08.925515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c5f9b-b959-477d-9e5c-6b8e7b824a82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:08.925794 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165c5f9b-b959-477d-9e5c-6b8e7b824a82 admin admin] PATH_INFO: `/v3/projects/1e7b344d68134e508864a9c6d6dabfb3` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:08.927004 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c5f9b-b959-477d-9e5c-6b8e7b824a82 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e7b344d68134e508864a9c6d6dabfb3)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:08.929191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-165c5f9b-b959-477d-9e5c-6b8e7b824a82 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:08.975891 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-165c5f9b-b959-477d-9e5c-6b8e7b824a82 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e7b344d68134e508864a9c6d6dabfb3', 'request_id': 'req-165c5f9b-b959-477d-9e5c-6b8e7b824a82'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:08.981837 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-165c5f9b-b959-477d-9e5c-6b8e7b824a82 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e7b344d68134e508864a9c6d6dabfb3', 'request_id': 'req-165c5f9b-b959-477d-9e5c-6b8e7b824a82'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:08.987993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-165c5f9b-b959-477d-9e5c-6b8e7b824a82 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e7b344d68134e508864a9c6d6dabfb3', 'request_id': 'req-165c5f9b-b959-477d-9e5c-6b8e7b824a82'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:09.000953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-165c5f9b-b959-477d-9e5c-6b8e7b824a82 admin admin] The token cache is being invalidate because project 1e7b344d68134e508864a9c6d6dabfb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:06:09.000953 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-165c5f9b-b959-477d-9e5c-6b8e7b824a82 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:09.001730 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-165c5f9b-b959-477d-9e5c-6b8e7b824a82 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:09.003442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1140/2302] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:06:08 2022] DELETE /identity/v3/projects/1e7b344d68134e508864a9c6d6dabfb3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:09.007452 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fcbfc-98a8-40a1-8b7b-4031ccd05591 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:09.008840 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a51fcbfc-98a8-40a1-8b7b-4031ccd05591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:09.023524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51fcbfc-98a8-40a1-8b7b-4031ccd05591 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:09.101308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f990e74-2560-4305-b078-7b20646973f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:09.102222 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f990e74-2560-4305-b078-7b20646973f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:09.102511 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f990e74-2560-4305-b078-7b20646973f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:09.102826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f990e74-2560-4305-b078-7b20646973f2 admin admin] PATH_INFO: `/v3/users/7dd47b2b7dc945418e477c75d640ad39` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:09.104318 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f990e74-2560-4305-b078-7b20646973f2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7dd47b2b7dc945418e477c75d640ad39)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:09.106313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f990e74-2560-4305-b078-7b20646973f2 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:09.169894 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3f990e74-2560-4305-b078-7b20646973f2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7dd47b2b7dc945418e477c75d640ad39'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:09.178507 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3f990e74-2560-4305-b078-7b20646973f2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7dd47b2b7dc945418e477c75d640ad39'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:09.179253 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3f990e74-2560-4305-b078-7b20646973f2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7dd47b2b7dc945418e477c75d640ad39'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:09.193072 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3f990e74-2560-4305-b078-7b20646973f2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7dd47b2b7dc945418e477c75d640ad39'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:09.193763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3f990e74-2560-4305-b078-7b20646973f2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7dd47b2b7dc945418e477c75d640ad39'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:09.207563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1163/2303] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:06:09 2022] DELETE /identity/v3/users/7dd47b2b7dc945418e477c75d640ad39 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:09.500444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c5f9b-b959-477d-9e5c-6b8e7b824a82 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:09.502546 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-165c5f9b-b959-477d-9e5c-6b8e7b824a82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:09.515915 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165c5f9b-b959-477d-9e5c-6b8e7b824a82 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:09.520021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:09.585170 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd735130-4457-4a30-8281-94a0dfa2e6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:09.586051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd735130-4457-4a30-8281-94a0dfa2e6cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:09.586341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd735130-4457-4a30-8281-94a0dfa2e6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:09.586644 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd735130-4457-4a30-8281-94a0dfa2e6cb admin admin] PATH_INFO: `/v3/projects/953852ace11742cc96f09875d8aa3901` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:09.587839 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd735130-4457-4a30-8281-94a0dfa2e6cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=953852ace11742cc96f09875d8aa3901)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:09.589757 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd735130-4457-4a30-8281-94a0dfa2e6cb admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:09.640087 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-dd735130-4457-4a30-8281-94a0dfa2e6cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '953852ace11742cc96f09875d8aa3901', 'request_id': 'req-dd735130-4457-4a30-8281-94a0dfa2e6cb'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:09.646501 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-dd735130-4457-4a30-8281-94a0dfa2e6cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '953852ace11742cc96f09875d8aa3901', 'request_id': 'req-dd735130-4457-4a30-8281-94a0dfa2e6cb'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:09.652777 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-dd735130-4457-4a30-8281-94a0dfa2e6cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '953852ace11742cc96f09875d8aa3901', 'request_id': 'req-dd735130-4457-4a30-8281-94a0dfa2e6cb'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:09.666893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-dd735130-4457-4a30-8281-94a0dfa2e6cb admin admin] The token cache is being invalidate because project 953852ace11742cc96f09875d8aa3901 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:06:09.667211 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-dd735130-4457-4a30-8281-94a0dfa2e6cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:09.667846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-dd735130-4457-4a30-8281-94a0dfa2e6cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:09.669389 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1141/2304] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:06:09 2022] DELETE /identity/v3/projects/953852ace11742cc96f09875d8aa3901 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:09.675837 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb29ed3-f35b-480c-9d92-6c81437db989 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:09.676344 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb29ed3-f35b-480c-9d92-6c81437db989 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:09.676461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb29ed3-f35b-480c-9d92-6c81437db989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:09.695867 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-3eb29ed3-f35b-480c-9d92-6c81437db989 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:09.697815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-3eb29ed3-f35b-480c-9d92-6c81437db989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:09.764218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1164/2305] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:09.773161 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd25c79-7057-4686-94e0-85d0fcdca7dd None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:09.773449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd25c79-7057-4686-94e0-85d0fcdca7dd None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:09.773726 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd25c79-7057-4686-94e0-85d0fcdca7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:09.792449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-6fd25c79-7057-4686-94e0-85d0fcdca7dd None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:09.793769 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6fd25c79-7057-4686-94e0-85d0fcdca7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:09.803149 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1142/2306] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:09.807524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb29ed3-f35b-480c-9d92-6c81437db989 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:09.818011 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb29ed3-f35b-480c-9d92-6c81437db989 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:09.822922 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93ff9f-c352-4d6c-a42d-6079950f6c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:09.823659 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93ff9f-c352-4d6c-a42d-6079950f6c4f admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:09.824015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93ff9f-c352-4d6c-a42d-6079950f6c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:09.824377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e93ff9f-c352-4d6c-a42d-6079950f6c4f admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:09.825189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93ff9f-c352-4d6c-a42d-6079950f6c4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:06:09.825566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93ff9f-c352-4d6c-a42d-6079950f6c4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:09.829106 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e93ff9f-c352-4d6c-a42d-6079950f6c4f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:09.829106 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:09.845928 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1165/2307] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:06:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:06:09.846803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd25c79-7057-4686-94e0-85d0fcdca7dd None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:09.855985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd25c79-7057-4686-94e0-85d0fcdca7dd None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:09.860817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602c643-03ab-4192-942b-6e408184c64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:09.861791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602c643-03ab-4192-942b-6e408184c64e admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:09.862080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602c643-03ab-4192-942b-6e408184c64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:09.862361 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3602c643-03ab-4192-942b-6e408184c64e admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:09.863116 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602c643-03ab-4192-942b-6e408184c64e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:09.865366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3602c643-03ab-4192-942b-6e408184c64e admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:09.899322 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1143/2308] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:06:09 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:06:09.900556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93ff9f-c352-4d6c-a42d-6079950f6c4f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:09.912536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e93ff9f-c352-4d6c-a42d-6079950f6c4f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:09.975217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4223a-8572-4555-baab-b49a1b722191 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:09.975897 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4223a-8572-4555-baab-b49a1b722191 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:09.976188 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4223a-8572-4555-baab-b49a1b722191 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:09.976470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab4223a-8572-4555-baab-b49a1b722191 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:09.977154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4223a-8572-4555-baab-b49a1b722191 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:09.979079 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab4223a-8572-4555-baab-b49a1b722191 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:10.003112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1166/2309] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:06:09 2022] POST /identity/v3/users => generated 291 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:06:10.007471 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602c643-03ab-4192-942b-6e408184c64e admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:10.017822 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3602c643-03ab-4192-942b-6e408184c64e admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:10.025144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e5526-3a19-4ecb-8b16-30d6d69d3045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:10.025867 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e5526-3a19-4ecb-8b16-30d6d69d3045 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:10.026203 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e5526-3a19-4ecb-8b16-30d6d69d3045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:10.026491 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e5526-3a19-4ecb-8b16-30d6d69d3045 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:10.027256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e5526-3a19-4ecb-8b16-30d6d69d3045 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:10.029040 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e5526-3a19-4ecb-8b16-30d6d69d3045 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:10.044205 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1144/2310] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:06:10.045940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4223a-8572-4555-baab-b49a1b722191 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:10.062155 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab4223a-8572-4555-baab-b49a1b722191 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:10.067664 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89343d60-88b8-41f6-8a6a-bbcb7cccd633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:10.068481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89343d60-88b8-41f6-8a6a-bbcb7cccd633 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:10.068646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89343d60-88b8-41f6-8a6a-bbcb7cccd633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:10.069064 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89343d60-88b8-41f6-8a6a-bbcb7cccd633 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:10.069859 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89343d60-88b8-41f6-8a6a-bbcb7cccd633 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:10.071698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89343d60-88b8-41f6-8a6a-bbcb7cccd633 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:10.082979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1167/2311] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:06:10.087734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e5526-3a19-4ecb-8b16-30d6d69d3045 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:10.097988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e5526-3a19-4ecb-8b16-30d6d69d3045 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:10.102535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982df69-f803-408c-b7fc-786f102ab9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:10.103597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982df69-f803-408c-b7fc-786f102ab9f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:10.103883 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982df69-f803-408c-b7fc-786f102ab9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:10.104161 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f982df69-f803-408c-b7fc-786f102ab9f1 admin admin] PATH_INFO: `/v3/projects/016b8f1fbeb24cda937a285b21db0c26/users/41fb0a133e5a4d66a1c8b1b799031c00/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:10.122322 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982df69-f803-408c-b7fc-786f102ab9f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=016b8f1fbeb24cda937a285b21db0c26, user_id=41fb0a133e5a4d66a1c8b1b799031c00, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:10.125091 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f982df69-f803-408c-b7fc-786f102ab9f1 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:10.140051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1145/2312] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:06:10 2022] PUT /identity/v3/projects/016b8f1fbeb24cda937a285b21db0c26/users/41fb0a133e5a4d66a1c8b1b799031c00/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:10.152120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365e618-3ba1-4564-9052-5a6a3da424d6 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:10.152715 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365e618-3ba1-4564-9052-5a6a3da424d6 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:10.153025 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e365e618-3ba1-4564-9052-5a6a3da424d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:10.188091 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-e365e618-3ba1-4564-9052-5a6a3da424d6 None None] MFA Rules not processed for user `41fb0a133e5a4d66a1c8b1b799031c00`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:10.188091 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e365e618-3ba1-4564-9052-5a6a3da424d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:10.262838 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1168/2313] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:06:10.267265 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982df69-f803-408c-b7fc-786f102ab9f1 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:10.304524 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:10.337171 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f982df69-f803-408c-b7fc-786f102ab9f1 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:10.341832 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971d0c3-266a-4f27-be46-05bd730f8ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:10.342501 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971d0c3-266a-4f27-be46-05bd730f8ba9 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:10.342946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971d0c3-266a-4f27-be46-05bd730f8ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:10.343232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d971d0c3-266a-4f27-be46-05bd730f8ba9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:10.343934 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971d0c3-266a-4f27-be46-05bd730f8ba9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:10.345950 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d971d0c3-266a-4f27-be46-05bd730f8ba9 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:10.372707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1146/2314] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:06:10 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:06:10.376608 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365e618-3ba1-4564-9052-5a6a3da424d6 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:10.392010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e365e618-3ba1-4564-9052-5a6a3da424d6 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:10.450539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3875a0b-fde3-483e-8ea7-d3da46c122cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:10.451292 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3875a0b-fde3-483e-8ea7-d3da46c122cd admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:10.451586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3875a0b-fde3-483e-8ea7-d3da46c122cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:10.451870 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3875a0b-fde3-483e-8ea7-d3da46c122cd admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:10.452627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3875a0b-fde3-483e-8ea7-d3da46c122cd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:10.454572 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3875a0b-fde3-483e-8ea7-d3da46c122cd admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:10.462802 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:10.476193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1169/2315] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:06:10 2022] POST /identity/v3/users => generated 298 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:06:10.479982 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971d0c3-266a-4f27-be46-05bd730f8ba9 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:10.491085 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d971d0c3-266a-4f27-be46-05bd730f8ba9 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:10.495764 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802f1ff-8aa0-43e1-b5d0-e0c9ea5a7e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:10.496428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802f1ff-8aa0-43e1-b5d0-e0c9ea5a7e2c admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:10.496719 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802f1ff-8aa0-43e1-b5d0-e0c9ea5a7e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:10.497005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0802f1ff-8aa0-43e1-b5d0-e0c9ea5a7e2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:10.497728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802f1ff-8aa0-43e1-b5d0-e0c9ea5a7e2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:10.499369 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0802f1ff-8aa0-43e1-b5d0-e0c9ea5a7e2c admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:10.512114 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1147/2316] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:06:10.516279 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3875a0b-fde3-483e-8ea7-d3da46c122cd admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:10.526477 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3875a0b-fde3-483e-8ea7-d3da46c122cd admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:10.531146 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99103a-1ccc-4b0a-a4c2-de702e51b9bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:10.532032 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99103a-1ccc-4b0a-a4c2-de702e51b9bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:10.532322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99103a-1ccc-4b0a-a4c2-de702e51b9bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:10.532606 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec99103a-1ccc-4b0a-a4c2-de702e51b9bc admin admin] PATH_INFO: `/v3/projects/330e66c39a4741c8b1bf83aa202f18d3/users/b63f1c71c62448ce8a14307c0863b525/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:10.549688 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99103a-1ccc-4b0a-a4c2-de702e51b9bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=330e66c39a4741c8b1bf83aa202f18d3, user_id=b63f1c71c62448ce8a14307c0863b525, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:10.552329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec99103a-1ccc-4b0a-a4c2-de702e51b9bc admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:10.565998 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1170/2317] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:06:10 2022] PUT /identity/v3/projects/330e66c39a4741c8b1bf83aa202f18d3/users/b63f1c71c62448ce8a14307c0863b525/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:10.569948 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802f1ff-8aa0-43e1-b5d0-e0c9ea5a7e2c admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:10.580208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0802f1ff-8aa0-43e1-b5d0-e0c9ea5a7e2c admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:10.638477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca9493a-b2a5-445b-ad99-8698bbf8f47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:10.639328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9493a-b2a5-445b-ad99-8698bbf8f47c admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:10.639612 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9493a-b2a5-445b-ad99-8698bbf8f47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:10.639889 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca9493a-b2a5-445b-ad99-8698bbf8f47c admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:10.640625 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9493a-b2a5-445b-ad99-8698bbf8f47c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:10.642227 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca9493a-b2a5-445b-ad99-8698bbf8f47c admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:10.652433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1148/2318] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:10 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:06:10.656645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99103a-1ccc-4b0a-a4c2-de702e51b9bc admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:10.667487 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec99103a-1ccc-4b0a-a4c2-de702e51b9bc admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:10.672094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d0215-57e3-4d70-a119-55cc8e19120b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:10.672980 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d0215-57e3-4d70-a119-55cc8e19120b admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:10.673263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d0215-57e3-4d70-a119-55cc8e19120b admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:10.673547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302d0215-57e3-4d70-a119-55cc8e19120b admin admin] PATH_INFO: `/v3/projects/330e66c39a4741c8b1bf83aa202f18d3/users/b63f1c71c62448ce8a14307c0863b525/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:10.676105 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d0215-57e3-4d70-a119-55cc8e19120b admin admin] RBAC: Authorizing `identity:create_grant(project_id=330e66c39a4741c8b1bf83aa202f18d3, user_id=b63f1c71c62448ce8a14307c0863b525, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:10.678781 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302d0215-57e3-4d70-a119-55cc8e19120b admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:10.689400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1171/2319] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:06:10 2022] PUT /identity/v3/projects/330e66c39a4741c8b1bf83aa202f18d3/users/b63f1c71c62448ce8a14307c0863b525/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:10.699547 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f63100-50ab-4d6e-94b5-94734d4404cf None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:10.699932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f63100-50ab-4d6e-94b5-94734d4404cf None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:10.700223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f63100-50ab-4d6e-94b5-94734d4404cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:10.728690 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-f6f63100-50ab-4d6e-94b5-94734d4404cf None None] MFA Rules not processed for user `b63f1c71c62448ce8a14307c0863b525`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:10.729912 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f6f63100-50ab-4d6e-94b5-94734d4404cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:10.799494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1149/2320] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:10 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:10.812451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d0215-57e3-4d70-a119-55cc8e19120b admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:10.814563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-302d0215-57e3-4d70-a119-55cc8e19120b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:10.872658 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302d0215-57e3-4d70-a119-55cc8e19120b admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:10.876493 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe9d89-13f1-4836-bbf4-52a78a3690c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:10.877187 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe9d89-13f1-4836-bbf4-52a78a3690c9 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:10.877479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe9d89-13f1-4836-bbf4-52a78a3690c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:10.877761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe9d89-13f1-4836-bbf4-52a78a3690c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:10.887162 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe9d89-13f1-4836-bbf4-52a78a3690c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:10.888597 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe9d89-13f1-4836-bbf4-52a78a3690c9 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:10.898360 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1172/2321] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:10 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:06:10.985401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f63100-50ab-4d6e-94b5-94734d4404cf None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:10.986855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f6f63100-50ab-4d6e-94b5-94734d4404cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:10.998508 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f63100-50ab-4d6e-94b5-94734d4404cf None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:11.002147 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8369578-0693-4b63-b99c-bf76f2e400be service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:11.002811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8369578-0693-4b63-b99c-bf76f2e400be service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:11.003100 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8369578-0693-4b63-b99c-bf76f2e400be service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:11.003380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8369578-0693-4b63-b99c-bf76f2e400be service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:11.029209 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:11.059456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8369578-0693-4b63-b99c-bf76f2e400be service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:11.060816 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8369578-0693-4b63-b99c-bf76f2e400be service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:11.070315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1150/2322] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:10 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:11.342696 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe9d89-13f1-4836-bbf4-52a78a3690c9 service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:11.421198 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe9d89-13f1-4836-bbf4-52a78a3690c9 service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:11.426262 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefd8c3-db2c-45c4-a27e-89a9aedafd53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:11.427167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefd8c3-db2c-45c4-a27e-89a9aedafd53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:11.427459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefd8c3-db2c-45c4-a27e-89a9aedafd53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:11.427744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeefd8c3-db2c-45c4-a27e-89a9aedafd53 admin admin] PATH_INFO: `/v3/users/41fb0a133e5a4d66a1c8b1b799031c00` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:11.428971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefd8c3-db2c-45c4-a27e-89a9aedafd53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41fb0a133e5a4d66a1c8b1b799031c00)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:11.430933 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeefd8c3-db2c-45c4-a27e-89a9aedafd53 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:11.480678 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-eeefd8c3-db2c-45c4-a27e-89a9aedafd53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41fb0a133e5a4d66a1c8b1b799031c00'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:11.488268 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-eeefd8c3-db2c-45c4-a27e-89a9aedafd53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41fb0a133e5a4d66a1c8b1b799031c00'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:11.488947 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-eeefd8c3-db2c-45c4-a27e-89a9aedafd53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41fb0a133e5a4d66a1c8b1b799031c00'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:11.490824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:11.503493 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-eeefd8c3-db2c-45c4-a27e-89a9aedafd53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41fb0a133e5a4d66a1c8b1b799031c00'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:11.504165 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-eeefd8c3-db2c-45c4-a27e-89a9aedafd53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41fb0a133e5a4d66a1c8b1b799031c00'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:11.517284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1173/2323] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:06:11 2022] DELETE /identity/v3/users/41fb0a133e5a4d66a1c8b1b799031c00 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:11.523399 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525594ea-a1ca-4b57-969f-c87f369a3f27 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:11.523900 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525594ea-a1ca-4b57-969f-c87f369a3f27 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:11.524305 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-525594ea-a1ca-4b57-969f-c87f369a3f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:11.542300 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-525594ea-a1ca-4b57-969f-c87f369a3f27 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:11.543890 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-525594ea-a1ca-4b57-969f-c87f369a3f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:11.615719 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1151/2324] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:11.625131 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefd8c3-db2c-45c4-a27e-89a9aedafd53 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:11.626702 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-eeefd8c3-db2c-45c4-a27e-89a9aedafd53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:11.686477 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeefd8c3-db2c-45c4-a27e-89a9aedafd53 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:11.690940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a76152-5a6b-4d70-a900-7276b91bd03e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:11.691615 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a76152-5a6b-4d70-a900-7276b91bd03e service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:11.691898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a76152-5a6b-4d70-a900-7276b91bd03e service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:11.692201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a76152-5a6b-4d70-a900-7276b91bd03e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:11.701178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a76152-5a6b-4d70-a900-7276b91bd03e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:11.702604 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a76152-5a6b-4d70-a900-7276b91bd03e service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:11.713508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1174/2325] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:11.990774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525594ea-a1ca-4b57-969f-c87f369a3f27 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:11.992223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-525594ea-a1ca-4b57-969f-c87f369a3f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:12.011034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-525594ea-a1ca-4b57-969f-c87f369a3f27 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:12.017668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed493bf4-4d17-40c0-b7b0-f61df03894c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:12.018737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed493bf4-4d17-40c0-b7b0-f61df03894c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:12.019032 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed493bf4-4d17-40c0-b7b0-f61df03894c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:12.019317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed493bf4-4d17-40c0-b7b0-f61df03894c7 admin admin] PATH_INFO: `/v3/projects/016b8f1fbeb24cda937a285b21db0c26` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:12.021860 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed493bf4-4d17-40c0-b7b0-f61df03894c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=016b8f1fbeb24cda937a285b21db0c26)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:12.024042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed493bf4-4d17-40c0-b7b0-f61df03894c7 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:12.081910 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-ed493bf4-4d17-40c0-b7b0-f61df03894c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '016b8f1fbeb24cda937a285b21db0c26', 'request_id': 'req-ed493bf4-4d17-40c0-b7b0-f61df03894c7'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:12.088344 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-ed493bf4-4d17-40c0-b7b0-f61df03894c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '016b8f1fbeb24cda937a285b21db0c26', 'request_id': 'req-ed493bf4-4d17-40c0-b7b0-f61df03894c7'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:12.094745 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-ed493bf4-4d17-40c0-b7b0-f61df03894c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '016b8f1fbeb24cda937a285b21db0c26', 'request_id': 'req-ed493bf4-4d17-40c0-b7b0-f61df03894c7'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:12.108159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-ed493bf4-4d17-40c0-b7b0-f61df03894c7 admin admin] The token cache is being invalidate because project 016b8f1fbeb24cda937a285b21db0c26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:06:12.108378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-ed493bf4-4d17-40c0-b7b0-f61df03894c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:12.109015 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-ed493bf4-4d17-40c0-b7b0-f61df03894c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:12.110580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1152/2326] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:06:11 2022] DELETE /identity/v3/projects/016b8f1fbeb24cda937a285b21db0c26 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:12.115099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a76152-5a6b-4d70-a900-7276b91bd03e service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:12.125077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a76152-5a6b-4d70-a900-7276b91bd03e service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:12.188895 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26aa93-14a2-42a9-8acc-476fc1056bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:12.189796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26aa93-14a2-42a9-8acc-476fc1056bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:12.190078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26aa93-14a2-42a9-8acc-476fc1056bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:12.190351 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc26aa93-14a2-42a9-8acc-476fc1056bc6 admin admin] PATH_INFO: `/v3/users/b63f1c71c62448ce8a14307c0863b525` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:12.191696 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26aa93-14a2-42a9-8acc-476fc1056bc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b63f1c71c62448ce8a14307c0863b525)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:12.193639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc26aa93-14a2-42a9-8acc-476fc1056bc6 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:12.246644 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-fc26aa93-14a2-42a9-8acc-476fc1056bc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b63f1c71c62448ce8a14307c0863b525'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:12.256158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-fc26aa93-14a2-42a9-8acc-476fc1056bc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b63f1c71c62448ce8a14307c0863b525'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:12.256864 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-fc26aa93-14a2-42a9-8acc-476fc1056bc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b63f1c71c62448ce8a14307c0863b525'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:12.269828 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-fc26aa93-14a2-42a9-8acc-476fc1056bc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b63f1c71c62448ce8a14307c0863b525'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:12.270489 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-fc26aa93-14a2-42a9-8acc-476fc1056bc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b63f1c71c62448ce8a14307c0863b525'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:12.279549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:12.286021 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1175/2327] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:06:12 2022] DELETE /identity/v3/users/b63f1c71c62448ce8a14307c0863b525 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:12.863250 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed493bf4-4d17-40c0-b7b0-f61df03894c7 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:12.864852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-ed493bf4-4d17-40c0-b7b0-f61df03894c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:12.878406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed493bf4-4d17-40c0-b7b0-f61df03894c7 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:12.941445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:12.953983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0405e9a8-f234-4747-b1c6-819c69ebadbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:12.955850 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405e9a8-f234-4747-b1c6-819c69ebadbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:12.956270 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405e9a8-f234-4747-b1c6-819c69ebadbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:12.957138 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0405e9a8-f234-4747-b1c6-819c69ebadbf admin admin] PATH_INFO: `/v3/projects/330e66c39a4741c8b1bf83aa202f18d3` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:12.959501 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0405e9a8-f234-4747-b1c6-819c69ebadbf admin admin] RBAC: Authorizing `identity:delete_project(project_id=330e66c39a4741c8b1bf83aa202f18d3)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:12.961861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0405e9a8-f234-4747-b1c6-819c69ebadbf admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:13.025705 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0405e9a8-f234-4747-b1c6-819c69ebadbf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '330e66c39a4741c8b1bf83aa202f18d3', 'request_id': 'req-0405e9a8-f234-4747-b1c6-819c69ebadbf'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:13.032835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0405e9a8-f234-4747-b1c6-819c69ebadbf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '330e66c39a4741c8b1bf83aa202f18d3', 'request_id': 'req-0405e9a8-f234-4747-b1c6-819c69ebadbf'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:13.039580 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0405e9a8-f234-4747-b1c6-819c69ebadbf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '330e66c39a4741c8b1bf83aa202f18d3', 'request_id': 'req-0405e9a8-f234-4747-b1c6-819c69ebadbf'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:13.070734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0405e9a8-f234-4747-b1c6-819c69ebadbf admin admin] The token cache is being invalidate because project 330e66c39a4741c8b1bf83aa202f18d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:06:13.071279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0405e9a8-f234-4747-b1c6-819c69ebadbf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:13.072321 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26aa93-14a2-42a9-8acc-476fc1056bc6 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:13.072792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0405e9a8-f234-4747-b1c6-819c69ebadbf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:13.074912 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1153/2328] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:06:12 2022] DELETE /identity/v3/projects/330e66c39a4741c8b1bf83aa202f18d3 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:13.076556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-fc26aa93-14a2-42a9-8acc-476fc1056bc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:13.085425 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916318d-95e7-4c32-be84-e9f2624a9153 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:13.086020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916318d-95e7-4c32-be84-e9f2624a9153 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:13.087030 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7916318d-95e7-4c32-be84-e9f2624a9153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:13.166028 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-7916318d-95e7-4c32-be84-e9f2624a9153 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:13.172524 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-7916318d-95e7-4c32-be84-e9f2624a9153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:13.200775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc26aa93-14a2-42a9-8acc-476fc1056bc6 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:13.281944 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1154/2329] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:13.290800 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df0497-f4ab-4466-be5e-2b211c5f432d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:13.290800 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df0497-f4ab-4466-be5e-2b211c5f432d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:13.291176 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df0497-f4ab-4466-be5e-2b211c5f432d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:13.309086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c91fe-5f8a-47a2-8964-cccec2aaf55d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:13.310251 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c91fe-5f8a-47a2-8964-cccec2aaf55d service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:13.310251 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c91fe-5f8a-47a2-8964-cccec2aaf55d service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:13.310454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497c91fe-5f8a-47a2-8964-cccec2aaf55d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:13.311297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-98df0497-f4ab-4466-be5e-2b211c5f432d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:13.312247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-497c91fe-5f8a-47a2-8964-cccec2aaf55d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:13.312469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-98df0497-f4ab-4466-be5e-2b211c5f432d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:13.321413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1155/2330] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:13.325801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df0497-f4ab-4466-be5e-2b211c5f432d None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:13.326569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c91fe-5f8a-47a2-8964-cccec2aaf55d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:13.328190 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497c91fe-5f8a-47a2-8964-cccec2aaf55d service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:13.334727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df0497-f4ab-4466-be5e-2b211c5f432d None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:13.339640 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1176/2331] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:13.345615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39351a-fefa-4920-9536-cc2e2a2c80a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:13.346157 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39351a-fefa-4920-9536-cc2e2a2c80a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:13.346448 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39351a-fefa-4920-9536-cc2e2a2c80a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:13.346789 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39351a-fefa-4920-9536-cc2e2a2c80a3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:13.347526 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39351a-fefa-4920-9536-cc2e2a2c80a3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:06:13.347837 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39351a-fefa-4920-9536-cc2e2a2c80a3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:13.349680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e39351a-fefa-4920-9536-cc2e2a2c80a3 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:13.362535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1156/2332] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 23:06:13.367020 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c91fe-5f8a-47a2-8964-cccec2aaf55d service nova] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:13.381549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497c91fe-5f8a-47a2-8964-cccec2aaf55d service nova] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:13.386119 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec14c8-35b9-4dbb-a7cc-4fec191308af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:13.386814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec14c8-35b9-4dbb-a7cc-4fec191308af admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:13.387100 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec14c8-35b9-4dbb-a7cc-4fec191308af admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:13.387379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec14c8-35b9-4dbb-a7cc-4fec191308af admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:13.388203 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec14c8-35b9-4dbb-a7cc-4fec191308af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:13.390100 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec14c8-35b9-4dbb-a7cc-4fec191308af admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:13.418327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1177/2333] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:06:13 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:06:13.419436 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39351a-fefa-4920-9536-cc2e2a2c80a3 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:13.431938 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39351a-fefa-4920-9536-cc2e2a2c80a3 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:13.488504 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454324e-077f-4b07-ac57-b0a0843f58b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:13.489184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454324e-077f-4b07-ac57-b0a0843f58b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:13.489475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454324e-077f-4b07-ac57-b0a0843f58b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:13.489759 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b454324e-077f-4b07-ac57-b0a0843f58b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:13.490443 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454324e-077f-4b07-ac57-b0a0843f58b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:13.492403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b454324e-077f-4b07-ac57-b0a0843f58b2 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:13.499916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:13.515563 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1157/2334] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:06:13 2022] POST /identity/v3/users => generated 287 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:06:13.519167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec14c8-35b9-4dbb-a7cc-4fec191308af admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:13.529546 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec14c8-35b9-4dbb-a7cc-4fec191308af admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:13.534785 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290b9a1-e662-4c73-8a4f-b7c2612c196c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:13.535518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290b9a1-e662-4c73-8a4f-b7c2612c196c admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:13.535871 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290b9a1-e662-4c73-8a4f-b7c2612c196c admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:13.536219 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290b9a1-e662-4c73-8a4f-b7c2612c196c admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:13.537023 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290b9a1-e662-4c73-8a4f-b7c2612c196c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:13.539115 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290b9a1-e662-4c73-8a4f-b7c2612c196c admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:13.551235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1178/2335] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:06:13.555201 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454324e-077f-4b07-ac57-b0a0843f58b2 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:13.566547 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b454324e-077f-4b07-ac57-b0a0843f58b2 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:13.571677 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b975f2d-86f1-45e7-a694-b861aa170053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:13.572527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b975f2d-86f1-45e7-a694-b861aa170053 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:13.572646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b975f2d-86f1-45e7-a694-b861aa170053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:13.572935 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b975f2d-86f1-45e7-a694-b861aa170053 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:13.573670 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b975f2d-86f1-45e7-a694-b861aa170053 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:13.575483 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b975f2d-86f1-45e7-a694-b861aa170053 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:13.589527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1158/2336] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:06:13.593825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290b9a1-e662-4c73-8a4f-b7c2612c196c admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:13.604908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290b9a1-e662-4c73-8a4f-b7c2612c196c admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:13.609832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f1993-11cd-4d63-98fb-20c88b734e7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:13.610794 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f1993-11cd-4d63-98fb-20c88b734e7d admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:13.611088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f1993-11cd-4d63-98fb-20c88b734e7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:13.611362 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779f1993-11cd-4d63-98fb-20c88b734e7d admin admin] PATH_INFO: `/v3/projects/0278008418ba450397733f6e87ae22bc/users/70daac5e6b644c188f3e3b1b872761fb/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:13.633710 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f1993-11cd-4d63-98fb-20c88b734e7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0278008418ba450397733f6e87ae22bc, user_id=70daac5e6b644c188f3e3b1b872761fb, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:13.633710 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779f1993-11cd-4d63-98fb-20c88b734e7d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:13.650325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1179/2337] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:06:13 2022] PUT /identity/v3/projects/0278008418ba450397733f6e87ae22bc/users/70daac5e6b644c188f3e3b1b872761fb/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:06:13.661017 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6bfa2-a130-45ad-b4f3-51ceecc69f7e None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:13.661317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6bfa2-a130-45ad-b4f3-51ceecc69f7e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:13.661598 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6bfa2-a130-45ad-b4f3-51ceecc69f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:13.698959 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-6fb6bfa2-a130-45ad-b4f3-51ceecc69f7e None None] MFA Rules not processed for user `70daac5e6b644c188f3e3b1b872761fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:13.700809 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6fb6bfa2-a130-45ad-b4f3-51ceecc69f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:13.782724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1159/2338] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:13.791782 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f1993-11cd-4d63-98fb-20c88b734e7d admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:13.794188 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-779f1993-11cd-4d63-98fb-20c88b734e7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:13.825315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:13.860434 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779f1993-11cd-4d63-98fb-20c88b734e7d admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:13.865085 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95b9ae-dfeb-4144-bf43-60c2af38704f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:13.865574 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95b9ae-dfeb-4144-bf43-60c2af38704f service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:13.865862 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95b9ae-dfeb-4144-bf43-60c2af38704f service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:13.866147 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a95b9ae-dfeb-4144-bf43-60c2af38704f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:13.876179 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95b9ae-dfeb-4144-bf43-60c2af38704f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:13.878171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a95b9ae-dfeb-4144-bf43-60c2af38704f service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:13.887424 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1180/2339] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:13 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:15.810280 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f24bf7-bd93-491c-bd6e-29cf36b07394 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:15.810797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f24bf7-bd93-491c-bd6e-29cf36b07394 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:15.810925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f24bf7-bd93-491c-bd6e-29cf36b07394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:15.829258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-f5f24bf7-bd93-491c-bd6e-29cf36b07394 None None] MFA Rules not processed for user `9e3a2d16133b4af8a208682de8a7d111`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:15.831894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f5f24bf7-bd93-491c-bd6e-29cf36b07394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:15.892980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1160/2340] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:16.216329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95b9ae-dfeb-4144-bf43-60c2af38704f service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:16.217892 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-0a95b9ae-dfeb-4144-bf43-60c2af38704f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:16.288764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a95b9ae-dfeb-4144-bf43-60c2af38704f service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:16.292368 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc3a1d-e01e-4edd-9179-beb6469fa0f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:16.293023 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc3a1d-e01e-4edd-9179-beb6469fa0f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:16.293315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc3a1d-e01e-4edd-9179-beb6469fa0f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:16.293599 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efc3a1d-e01e-4edd-9179-beb6469fa0f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:16.295649 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-0efc3a1d-e01e-4edd-9179-beb6469fa0f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:16.305826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc3a1d-e01e-4edd-9179-beb6469fa0f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:16.307600 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efc3a1d-e01e-4edd-9179-beb6469fa0f4 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:16.318048 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1181/2341] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:06:16.359757 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684f1cf-b5bd-4527-b82d-629943a33acb None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:16.360161 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684f1cf-b5bd-4527-b82d-629943a33acb None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:16.360478 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4684f1cf-b5bd-4527-b82d-629943a33acb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:16.380166 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-4684f1cf-b5bd-4527-b82d-629943a33acb None None] MFA Rules not processed for user `2697aa77265e4f5783f87ffb3d4992b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:16.381397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-4684f1cf-b5bd-4527-b82d-629943a33acb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:16.440184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1161/2342] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:16.848348 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc3a1d-e01e-4edd-9179-beb6469fa0f4 service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:16.857493 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efc3a1d-e01e-4edd-9179-beb6469fa0f4 service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:16.861454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974003e4-3d08-4c80-8912-2387c0a03d1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:16.862112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974003e4-3d08-4c80-8912-2387c0a03d1d service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:16.862402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974003e4-3d08-4c80-8912-2387c0a03d1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:16.862719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974003e4-3d08-4c80-8912-2387c0a03d1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:16.864590 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-974003e4-3d08-4c80-8912-2387c0a03d1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:16.878427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974003e4-3d08-4c80-8912-2387c0a03d1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:16.879934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974003e4-3d08-4c80-8912-2387c0a03d1d service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:16.890368 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1182/2343] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:17.054436 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684f1cf-b5bd-4527-b82d-629943a33acb None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:17.080484 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:17.137693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4684f1cf-b5bd-4527-b82d-629943a33acb None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:17.142282 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca979904-01de-47ce-82c3-4ee75dc733c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:17.143209 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca979904-01de-47ce-82c3-4ee75dc733c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:17.143498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca979904-01de-47ce-82c3-4ee75dc733c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:17.143778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca979904-01de-47ce-82c3-4ee75dc733c9 admin admin] PATH_INFO: `/v3/users/70daac5e6b644c188f3e3b1b872761fb` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:17.144972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca979904-01de-47ce-82c3-4ee75dc733c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70daac5e6b644c188f3e3b1b872761fb)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:17.148221 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca979904-01de-47ce-82c3-4ee75dc733c9 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:17.204604 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-ca979904-01de-47ce-82c3-4ee75dc733c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70daac5e6b644c188f3e3b1b872761fb'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:17.205389 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-ca979904-01de-47ce-82c3-4ee75dc733c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70daac5e6b644c188f3e3b1b872761fb'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:17.227912 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-ca979904-01de-47ce-82c3-4ee75dc733c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70daac5e6b644c188f3e3b1b872761fb'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:17.228706 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-ca979904-01de-47ce-82c3-4ee75dc733c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70daac5e6b644c188f3e3b1b872761fb'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:17.246478 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-ca979904-01de-47ce-82c3-4ee75dc733c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70daac5e6b644c188f3e3b1b872761fb'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:17.270686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1162/2344] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:06:17 2022] DELETE /identity/v3/users/70daac5e6b644c188f3e3b1b872761fb => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:17.275502 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f2544-d69a-4dd2-9a42-5f0225ac7e97 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:17.276025 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f2544-d69a-4dd2-9a42-5f0225ac7e97 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:17.276218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f2544-d69a-4dd2-9a42-5f0225ac7e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:17.294493 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-1b1f2544-d69a-4dd2-9a42-5f0225ac7e97 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:17.295754 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-1b1f2544-d69a-4dd2-9a42-5f0225ac7e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:17.362245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1183/2345] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:17.372388 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca979904-01de-47ce-82c3-4ee75dc733c9 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:17.373958 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-ca979904-01de-47ce-82c3-4ee75dc733c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:17.438782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca979904-01de-47ce-82c3-4ee75dc733c9 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:17.442634 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e46a23b-e52a-4579-9e60-f6354b4b057f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:17.445616 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46a23b-e52a-4579-9e60-f6354b4b057f service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:17.445616 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46a23b-e52a-4579-9e60-f6354b4b057f service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:17.445616 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e46a23b-e52a-4579-9e60-f6354b4b057f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:17.455440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46a23b-e52a-4579-9e60-f6354b4b057f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:17.457500 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e46a23b-e52a-4579-9e60-f6354b4b057f service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:17.469846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1163/2346] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:17.813135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f2544-d69a-4dd2-9a42-5f0225ac7e97 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:17.815333 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-1b1f2544-d69a-4dd2-9a42-5f0225ac7e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:17.829009 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f2544-d69a-4dd2-9a42-5f0225ac7e97 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:17.845636 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0a0230-46a9-4a4b-abc9-405087e7a528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:17.846890 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a0230-46a9-4a4b-abc9-405087e7a528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:17.847330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a0230-46a9-4a4b-abc9-405087e7a528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:17.848527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0a0230-46a9-4a4b-abc9-405087e7a528 admin admin] PATH_INFO: `/v3/projects/0278008418ba450397733f6e87ae22bc` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:17.850330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a0230-46a9-4a4b-abc9-405087e7a528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0278008418ba450397733f6e87ae22bc)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:17.852468 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0a0230-46a9-4a4b-abc9-405087e7a528 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:17.909683 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-ef0a0230-46a9-4a4b-abc9-405087e7a528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0278008418ba450397733f6e87ae22bc', 'request_id': 'req-ef0a0230-46a9-4a4b-abc9-405087e7a528'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:17.911839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:17.917021 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-ef0a0230-46a9-4a4b-abc9-405087e7a528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0278008418ba450397733f6e87ae22bc', 'request_id': 'req-ef0a0230-46a9-4a4b-abc9-405087e7a528'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:17.924009 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-ef0a0230-46a9-4a4b-abc9-405087e7a528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0278008418ba450397733f6e87ae22bc', 'request_id': 'req-ef0a0230-46a9-4a4b-abc9-405087e7a528'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:17.938925 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-ef0a0230-46a9-4a4b-abc9-405087e7a528 admin admin] The token cache is being invalidate because project 0278008418ba450397733f6e87ae22bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:06:17.939332 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-ef0a0230-46a9-4a4b-abc9-405087e7a528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:17.940072 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-ef0a0230-46a9-4a4b-abc9-405087e7a528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:17.941846 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1184/2347] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:06:17 2022] DELETE /identity/v3/projects/0278008418ba450397733f6e87ae22bc => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:17.947184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8181bb-6a59-4e3f-909e-bf4e1f0d96f1 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:17.947485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8181bb-6a59-4e3f-909e-bf4e1f0d96f1 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:17.948180 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8181bb-6a59-4e3f-909e-bf4e1f0d96f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:17.976222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-ea8181bb-6a59-4e3f-909e-bf4e1f0d96f1 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:17.978178 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-ea8181bb-6a59-4e3f-909e-bf4e1f0d96f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:18.023659 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:18.049397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1164/2348] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:18.056953 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2880923-1e33-4392-b225-b347bde27259 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:18.057292 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2880923-1e33-4392-b225-b347bde27259 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:18.057523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2880923-1e33-4392-b225-b347bde27259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:18.076834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-c2880923-1e33-4392-b225-b347bde27259 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:18.078036 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c2880923-1e33-4392-b225-b347bde27259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:18.086462 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1185/2349] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:18.093270 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8181bb-6a59-4e3f-909e-bf4e1f0d96f1 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:18.110870 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8181bb-6a59-4e3f-909e-bf4e1f0d96f1 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:18.119540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737af583-ffce-40df-a338-084165e36cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:18.120770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737af583-ffce-40df-a338-084165e36cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:18.121291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737af583-ffce-40df-a338-084165e36cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:18.121785 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737af583-ffce-40df-a338-084165e36cb5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:18.123077 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737af583-ffce-40df-a338-084165e36cb5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:06:18.123667 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737af583-ffce-40df-a338-084165e36cb5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:18.126000 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737af583-ffce-40df-a338-084165e36cb5 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:18.142109 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1165/2350] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:06:18.146317 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2880923-1e33-4392-b225-b347bde27259 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:18.159027 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2880923-1e33-4392-b225-b347bde27259 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:18.170772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04848fe4-4996-4f91-9230-8def243bd590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:18.170772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04848fe4-4996-4f91-9230-8def243bd590 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:18.170772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04848fe4-4996-4f91-9230-8def243bd590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:18.170772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04848fe4-4996-4f91-9230-8def243bd590 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:18.171212 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04848fe4-4996-4f91-9230-8def243bd590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:18.173099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04848fe4-4996-4f91-9230-8def243bd590 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:18.208855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1186/2351] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:06:18 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:06:18.210318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737af583-ffce-40df-a338-084165e36cb5 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:18.221611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737af583-ffce-40df-a338-084165e36cb5 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:18.282735 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9353227-640c-4aec-ae19-9eacfca8b77b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:18.283432 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9353227-640c-4aec-ae19-9eacfca8b77b admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:18.283720 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9353227-640c-4aec-ae19-9eacfca8b77b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:18.283998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9353227-640c-4aec-ae19-9eacfca8b77b admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:18.284684 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9353227-640c-4aec-ae19-9eacfca8b77b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:18.286864 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9353227-640c-4aec-ae19-9eacfca8b77b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:18.308696 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1166/2352] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:06:18 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:06:18.312912 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04848fe4-4996-4f91-9230-8def243bd590 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:18.324187 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04848fe4-4996-4f91-9230-8def243bd590 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:18.331499 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54555ad-201d-4010-a7bc-647927c06dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:18.332192 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54555ad-201d-4010-a7bc-647927c06dd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:18.332480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54555ad-201d-4010-a7bc-647927c06dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:18.332761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54555ad-201d-4010-a7bc-647927c06dd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:18.333501 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54555ad-201d-4010-a7bc-647927c06dd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:18.335161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54555ad-201d-4010-a7bc-647927c06dd2 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:18.350403 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1187/2353] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:06:18.352636 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9353227-640c-4aec-ae19-9eacfca8b77b admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:18.363499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9353227-640c-4aec-ae19-9eacfca8b77b admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:18.369511 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9fe610-11cd-4c14-aa8a-8c925b9cd1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:18.370197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fe610-11cd-4c14-aa8a-8c925b9cd1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:18.370811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fe610-11cd-4c14-aa8a-8c925b9cd1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:18.371098 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9fe610-11cd-4c14-aa8a-8c925b9cd1c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:18.371889 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9fe610-11cd-4c14-aa8a-8c925b9cd1c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:18.373825 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9fe610-11cd-4c14-aa8a-8c925b9cd1c1 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:18.387923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1167/2354] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:06:18.389382 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54555ad-201d-4010-a7bc-647927c06dd2 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:18.401158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54555ad-201d-4010-a7bc-647927c06dd2 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:18.405787 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a64a3e-053a-4071-81c3-0321c1767f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:18.406712 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a64a3e-053a-4071-81c3-0321c1767f9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:18.407004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a64a3e-053a-4071-81c3-0321c1767f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:18.407325 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a64a3e-053a-4071-81c3-0321c1767f9f admin admin] PATH_INFO: `/v3/projects/b9569a52dfd64754b5bfcae2423c8689/users/7cc9ca2fdc704b63a2049921accbe444/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:18.422175 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a64a3e-053a-4071-81c3-0321c1767f9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9569a52dfd64754b5bfcae2423c8689, user_id=7cc9ca2fdc704b63a2049921accbe444, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:18.425419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a64a3e-053a-4071-81c3-0321c1767f9f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:18.443539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1188/2355] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:06:18 2022] PUT /identity/v3/projects/b9569a52dfd64754b5bfcae2423c8689/users/7cc9ca2fdc704b63a2049921accbe444/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:18.454551 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca727a11-7a00-4617-ba81-56315174ef88 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:18.454892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca727a11-7a00-4617-ba81-56315174ef88 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:18.455188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca727a11-7a00-4617-ba81-56315174ef88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:18.485629 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-ca727a11-7a00-4617-ba81-56315174ef88 None None] MFA Rules not processed for user `7cc9ca2fdc704b63a2049921accbe444`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:18.486924 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-ca727a11-7a00-4617-ba81-56315174ef88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:18.567489 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1168/2356] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:18.571995 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a64a3e-053a-4071-81c3-0321c1767f9f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:18.656555 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a64a3e-053a-4071-81c3-0321c1767f9f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:18.665327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09f7ec-acad-45b1-959d-9fb1473203fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:18.666340 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09f7ec-acad-45b1-959d-9fb1473203fd admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:18.666680 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09f7ec-acad-45b1-959d-9fb1473203fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:18.667000 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09f7ec-acad-45b1-959d-9fb1473203fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:18.667870 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09f7ec-acad-45b1-959d-9fb1473203fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:18.670197 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09f7ec-acad-45b1-959d-9fb1473203fd admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:18.708808 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1189/2357] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:06:18 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:06:18.712828 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca727a11-7a00-4617-ba81-56315174ef88 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:18.729632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca727a11-7a00-4617-ba81-56315174ef88 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:18.773128 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:18.814709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d204c-1168-4c81-9528-cbc62ce75676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:18.815484 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d204c-1168-4c81-9528-cbc62ce75676 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:18.815775 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d204c-1168-4c81-9528-cbc62ce75676 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:18.816050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81d204c-1168-4c81-9528-cbc62ce75676 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:18.816741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d204c-1168-4c81-9528-cbc62ce75676 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:18.818941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81d204c-1168-4c81-9528-cbc62ce75676 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:18.856418 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09f7ec-acad-45b1-959d-9fb1473203fd admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:18.864070 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1169/2358] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:06:18 2022] POST /identity/v3/users => generated 292 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:06:18.868362 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09f7ec-acad-45b1-959d-9fb1473203fd admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:18.882315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6c5aa-b6c4-4f44-9101-6745c6b3322f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:18.883436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6c5aa-b6c4-4f44-9101-6745c6b3322f admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:18.884153 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6c5aa-b6c4-4f44-9101-6745c6b3322f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:18.884641 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6c5aa-b6c4-4f44-9101-6745c6b3322f admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:18.885930 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6c5aa-b6c4-4f44-9101-6745c6b3322f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:18.888278 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec6c5aa-b6c4-4f44-9101-6745c6b3322f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:18.910650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1190/2359] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:06:18.914534 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d204c-1168-4c81-9528-cbc62ce75676 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:18.926303 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81d204c-1168-4c81-9528-cbc62ce75676 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:18.934819 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17fb791-bbc3-4236-b551-332fe53e9eb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:18.935584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fb791-bbc3-4236-b551-332fe53e9eb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:18.936016 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fb791-bbc3-4236-b551-332fe53e9eb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:18.936356 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17fb791-bbc3-4236-b551-332fe53e9eb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:18.937531 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fb791-bbc3-4236-b551-332fe53e9eb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:18.939853 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17fb791-bbc3-4236-b551-332fe53e9eb8 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:18.955374 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1170/2360] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:18 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:06:18.956901 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6c5aa-b6c4-4f44-9101-6745c6b3322f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:18.969708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6c5aa-b6c4-4f44-9101-6745c6b3322f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:18.978645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a7aee-db90-478a-8935-0aeb1e95e9e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:18.979633 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a7aee-db90-478a-8935-0aeb1e95e9e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:18.980058 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a7aee-db90-478a-8935-0aeb1e95e9e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:18.980350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1a7aee-db90-478a-8935-0aeb1e95e9e4 admin admin] PATH_INFO: `/v3/projects/341857b9ea7045b2967612ab4659856f/users/dde5489066fb49a2b68b15bc62f53301/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:19.004815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a7aee-db90-478a-8935-0aeb1e95e9e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=341857b9ea7045b2967612ab4659856f, user_id=dde5489066fb49a2b68b15bc62f53301, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:19.007633 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1a7aee-db90-478a-8935-0aeb1e95e9e4 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:19.026128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1191/2361] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:06:18 2022] PUT /identity/v3/projects/341857b9ea7045b2967612ab4659856f/users/dde5489066fb49a2b68b15bc62f53301/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:19.036204 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be4891-1fbe-45a9-bffb-bca1fcaad9de None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:19.036558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be4891-1fbe-45a9-bffb-bca1fcaad9de None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:19.036902 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36be4891-1fbe-45a9-bffb-bca1fcaad9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:19.067620 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-36be4891-1fbe-45a9-bffb-bca1fcaad9de None None] MFA Rules not processed for user `dde5489066fb49a2b68b15bc62f53301`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:19.069056 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-36be4891-1fbe-45a9-bffb-bca1fcaad9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:19.181495 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1171/2362] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:19.185673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a7aee-db90-478a-8935-0aeb1e95e9e4 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:19.222401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:19.273827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1a7aee-db90-478a-8935-0aeb1e95e9e4 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:19.279359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397b6aa-16d9-4f98-aacf-9847949406f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:19.280054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397b6aa-16d9-4f98-aacf-9847949406f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:19.280345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397b6aa-16d9-4f98-aacf-9847949406f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:19.280645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f397b6aa-16d9-4f98-aacf-9847949406f2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:19.282622 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397b6aa-16d9-4f98-aacf-9847949406f2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:19.284503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f397b6aa-16d9-4f98-aacf-9847949406f2 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:19.323351 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1192/2363] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:06:19 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:06:19.324562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be4891-1fbe-45a9-bffb-bca1fcaad9de None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:19.337646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36be4891-1fbe-45a9-bffb-bca1fcaad9de None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:19.468335 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ce3ed-54c9-4a25-96e8-779c927e8c33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:19.469769 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ce3ed-54c9-4a25-96e8-779c927e8c33 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:19.470732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ce3ed-54c9-4a25-96e8-779c927e8c33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:19.471076 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8ce3ed-54c9-4a25-96e8-779c927e8c33 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:19.471803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ce3ed-54c9-4a25-96e8-779c927e8c33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:19.475654 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8ce3ed-54c9-4a25-96e8-779c927e8c33 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:19.508362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1172/2364] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:06:19 2022] POST /identity/v3/users => generated 299 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:06:19.508824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397b6aa-16d9-4f98-aacf-9847949406f2 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:19.532978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f397b6aa-16d9-4f98-aacf-9847949406f2 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:19.553333 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcead1-1fa8-48f9-a816-7a6f9ba602a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:19.554382 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcead1-1fa8-48f9-a816-7a6f9ba602a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:19.554800 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcead1-1fa8-48f9-a816-7a6f9ba602a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:19.561184 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffcead1-1fa8-48f9-a816-7a6f9ba602a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:19.562473 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcead1-1fa8-48f9-a816-7a6f9ba602a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:19.565950 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffcead1-1fa8-48f9-a816-7a6f9ba602a5 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:19.587151 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1193/2365] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:06:19.590835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ce3ed-54c9-4a25-96e8-779c927e8c33 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:19.617705 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8ce3ed-54c9-4a25-96e8-779c927e8c33 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:19.627983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef793c0f-3720-497e-a47c-5aa0c9a245f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:19.628924 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef793c0f-3720-497e-a47c-5aa0c9a245f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:19.629209 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef793c0f-3720-497e-a47c-5aa0c9a245f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:19.629481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef793c0f-3720-497e-a47c-5aa0c9a245f4 admin admin] PATH_INFO: `/v3/projects/7aaa6b56fcdb4735b485868f9dbfe574/users/c769a5edd7bf42f7a7df03c1be6580da/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:19.678093 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef793c0f-3720-497e-a47c-5aa0c9a245f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aaa6b56fcdb4735b485868f9dbfe574, user_id=c769a5edd7bf42f7a7df03c1be6580da, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:19.681881 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef793c0f-3720-497e-a47c-5aa0c9a245f4 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:19.722116 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1173/2366] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:06:19 2022] PUT /identity/v3/projects/7aaa6b56fcdb4735b485868f9dbfe574/users/c769a5edd7bf42f7a7df03c1be6580da/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:19.725999 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcead1-1fa8-48f9-a816-7a6f9ba602a5 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:19.744144 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffcead1-1fa8-48f9-a816-7a6f9ba602a5 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:19.857419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722df1bb-b57c-4b2a-a223-c9b61b2e8444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:19.858341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722df1bb-b57c-4b2a-a223-c9b61b2e8444 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:19.858895 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722df1bb-b57c-4b2a-a223-c9b61b2e8444 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:19.859230 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722df1bb-b57c-4b2a-a223-c9b61b2e8444 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:19.860417 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722df1bb-b57c-4b2a-a223-c9b61b2e8444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:19.863017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722df1bb-b57c-4b2a-a223-c9b61b2e8444 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:19.882529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1194/2367] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:19 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:06:19.883881 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef793c0f-3720-497e-a47c-5aa0c9a245f4 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:19.896139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef793c0f-3720-497e-a47c-5aa0c9a245f4 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:19.915585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ebedf-140a-4435-96d4-1deeec249fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:19.921858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ebedf-140a-4435-96d4-1deeec249fc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:19.922149 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ebedf-140a-4435-96d4-1deeec249fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:19.922452 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1ebedf-140a-4435-96d4-1deeec249fc0 admin admin] PATH_INFO: `/v3/projects/7aaa6b56fcdb4735b485868f9dbfe574/users/c769a5edd7bf42f7a7df03c1be6580da/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:19.931532 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ebedf-140a-4435-96d4-1deeec249fc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aaa6b56fcdb4735b485868f9dbfe574, user_id=c769a5edd7bf42f7a7df03c1be6580da, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:19.935108 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1ebedf-140a-4435-96d4-1deeec249fc0 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:19.952283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:19.966347 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1174/2368] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:06:19 2022] PUT /identity/v3/projects/7aaa6b56fcdb4735b485868f9dbfe574/users/c769a5edd7bf42f7a7df03c1be6580da/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:19.976905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c995ba1-2a0c-47c1-9872-91fe9872c9af None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:19.977753 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c995ba1-2a0c-47c1-9872-91fe9872c9af None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:19.978058 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c995ba1-2a0c-47c1-9872-91fe9872c9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:20.029654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-6c995ba1-2a0c-47c1-9872-91fe9872c9af None None] MFA Rules not processed for user `c769a5edd7bf42f7a7df03c1be6580da`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:20.031008 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-6c995ba1-2a0c-47c1-9872-91fe9872c9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:20.158479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1195/2369] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:20.178436 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ebedf-140a-4435-96d4-1deeec249fc0 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:20.179856 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6c1ebedf-140a-4435-96d4-1deeec249fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:20.275212 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1ebedf-140a-4435-96d4-1deeec249fc0 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:20.279988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2250eb0-aa54-46c1-8257-7fe53a7dcd73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:20.280930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2250eb0-aa54-46c1-8257-7fe53a7dcd73 service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:20.281157 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2250eb0-aa54-46c1-8257-7fe53a7dcd73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:20.281444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2250eb0-aa54-46c1-8257-7fe53a7dcd73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:20.373021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2250eb0-aa54-46c1-8257-7fe53a7dcd73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:20.374816 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2250eb0-aa54-46c1-8257-7fe53a7dcd73 service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:20.386224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1175/2370] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:20.486010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c995ba1-2a0c-47c1-9872-91fe9872c9af None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:20.487477 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-6c995ba1-2a0c-47c1-9872-91fe9872c9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:20.524916 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:20.597727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c995ba1-2a0c-47c1-9872-91fe9872c9af None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:20.604817 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bb855-c36c-46e5-83ae-9b777a5be4d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:20.605666 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bb855-c36c-46e5-83ae-9b777a5be4d8 service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:20.606046 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bb855-c36c-46e5-83ae-9b777a5be4d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:20.606435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476bb855-c36c-46e5-83ae-9b777a5be4d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:20.608623 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-476bb855-c36c-46e5-83ae-9b777a5be4d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:20.625353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bb855-c36c-46e5-83ae-9b777a5be4d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:20.627638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476bb855-c36c-46e5-83ae-9b777a5be4d8 service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:20.653767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1196/2371] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:22.012713 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2250eb0-aa54-46c1-8257-7fe53a7dcd73 service cinder] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:22.015711 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-a2250eb0-aa54-46c1-8257-7fe53a7dcd73 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:22.028234 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2250eb0-aa54-46c1-8257-7fe53a7dcd73 service cinder] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:22.032115 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719415ba-5596-428c-abee-d5d14115716b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:22.032772 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719415ba-5596-428c-abee-d5d14115716b service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:22.033061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719415ba-5596-428c-abee-d5d14115716b service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:22.033344 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719415ba-5596-428c-abee-d5d14115716b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:22.127362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719415ba-5596-428c-abee-d5d14115716b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:22.128784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719415ba-5596-428c-abee-d5d14115716b service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:22.149428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1176/2372] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:22 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:22.786305 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bb855-c36c-46e5-83ae-9b777a5be4d8 service nova] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:22.795996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476bb855-c36c-46e5-83ae-9b777a5be4d8 service nova] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:22.800956 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e63e1-d741-46f6-aa67-9b524f163cf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:22.801685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e63e1-d741-46f6-aa67-9b524f163cf3 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:22.802042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e63e1-d741-46f6-aa67-9b524f163cf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:22.802411 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e63e1-d741-46f6-aa67-9b524f163cf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:22.812187 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e63e1-d741-46f6-aa67-9b524f163cf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:22.813769 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e63e1-d741-46f6-aa67-9b524f163cf3 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:22.824239 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1197/2373] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:22 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:25.700514 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719415ba-5596-428c-abee-d5d14115716b service cinder] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:25.702446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-719415ba-5596-428c-abee-d5d14115716b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:25.720355 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719415ba-5596-428c-abee-d5d14115716b service cinder] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:25.726181 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74e2585-6510-45bf-bb0f-239de7d3d5b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:25.727128 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e2585-6510-45bf-bb0f-239de7d3d5b8 service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:25.727463 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e2585-6510-45bf-bb0f-239de7d3d5b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:25.727754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74e2585-6510-45bf-bb0f-239de7d3d5b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:25.729760 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-e74e2585-6510-45bf-bb0f-239de7d3d5b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:25.742888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e2585-6510-45bf-bb0f-239de7d3d5b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:25.744472 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74e2585-6510-45bf-bb0f-239de7d3d5b8 service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:25.756594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1177/2374] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:26.654218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e63e1-d741-46f6-aa67-9b524f163cf3 service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:26.716620 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e63e1-d741-46f6-aa67-9b524f163cf3 service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:26.721218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c80e0-69c0-4b48-a29c-b7b335a4d65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:26.722082 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c80e0-69c0-4b48-a29c-b7b335a4d65e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:26.722367 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c80e0-69c0-4b48-a29c-b7b335a4d65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:26.722673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787c80e0-69c0-4b48-a29c-b7b335a4d65e admin admin] PATH_INFO: `/v3/users/7cc9ca2fdc704b63a2049921accbe444` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:26.723850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c80e0-69c0-4b48-a29c-b7b335a4d65e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7cc9ca2fdc704b63a2049921accbe444)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:26.725775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787c80e0-69c0-4b48-a29c-b7b335a4d65e admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:26.770782 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-787c80e0-69c0-4b48-a29c-b7b335a4d65e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7cc9ca2fdc704b63a2049921accbe444'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:26.777537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-787c80e0-69c0-4b48-a29c-b7b335a4d65e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7cc9ca2fdc704b63a2049921accbe444'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:26.778190 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-787c80e0-69c0-4b48-a29c-b7b335a4d65e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7cc9ca2fdc704b63a2049921accbe444'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:26.790004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-787c80e0-69c0-4b48-a29c-b7b335a4d65e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7cc9ca2fdc704b63a2049921accbe444'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:26.790661 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-787c80e0-69c0-4b48-a29c-b7b335a4d65e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7cc9ca2fdc704b63a2049921accbe444'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:26.803730 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1198/2375] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:06:26 2022] DELETE /identity/v3/users/7cc9ca2fdc704b63a2049921accbe444 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:26.808196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc71da5-632a-42c9-90bd-6a12b067f904 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:26.808667 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc71da5-632a-42c9-90bd-6a12b067f904 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:26.808763 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc71da5-632a-42c9-90bd-6a12b067f904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:26.825960 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-dfc71da5-632a-42c9-90bd-6a12b067f904 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:26.827200 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-dfc71da5-632a-42c9-90bd-6a12b067f904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:26.865836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:26.892489 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1178/2376] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:26.902032 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c80e0-69c0-4b48-a29c-b7b335a4d65e admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:26.903474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-787c80e0-69c0-4b48-a29c-b7b335a4d65e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:26.919030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:26.962895 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787c80e0-69c0-4b48-a29c-b7b335a4d65e admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:26.966890 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20b378-332e-4653-8e41-13c24b3e545c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:26.967620 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20b378-332e-4653-8e41-13c24b3e545c service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:26.967908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20b378-332e-4653-8e41-13c24b3e545c service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:26.968191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c20b378-332e-4653-8e41-13c24b3e545c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:26.977941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20b378-332e-4653-8e41-13c24b3e545c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:26.979435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c20b378-332e-4653-8e41-13c24b3e545c service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:26.989739 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1199/2377] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:27.360926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc71da5-632a-42c9-90bd-6a12b067f904 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:27.362408 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-dfc71da5-632a-42c9-90bd-6a12b067f904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:27.374188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc71da5-632a-42c9-90bd-6a12b067f904 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:27.378891 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dd32c-aad6-4f50-a44d-af104c5d7eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:27.379694 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dd32c-aad6-4f50-a44d-af104c5d7eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:27.379981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dd32c-aad6-4f50-a44d-af104c5d7eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:27.380261 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925dd32c-aad6-4f50-a44d-af104c5d7eb5 admin admin] PATH_INFO: `/v3/projects/b9569a52dfd64754b5bfcae2423c8689` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:27.381440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925dd32c-aad6-4f50-a44d-af104c5d7eb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9569a52dfd64754b5bfcae2423c8689)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:27.383386 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925dd32c-aad6-4f50-a44d-af104c5d7eb5 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:27.437333 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-925dd32c-aad6-4f50-a44d-af104c5d7eb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9569a52dfd64754b5bfcae2423c8689', 'request_id': 'req-925dd32c-aad6-4f50-a44d-af104c5d7eb5'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:27.444391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-925dd32c-aad6-4f50-a44d-af104c5d7eb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9569a52dfd64754b5bfcae2423c8689', 'request_id': 'req-925dd32c-aad6-4f50-a44d-af104c5d7eb5'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:27.451144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-925dd32c-aad6-4f50-a44d-af104c5d7eb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9569a52dfd64754b5bfcae2423c8689', 'request_id': 'req-925dd32c-aad6-4f50-a44d-af104c5d7eb5'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:27.466759 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-925dd32c-aad6-4f50-a44d-af104c5d7eb5 admin admin] The token cache is being invalidate because project b9569a52dfd64754b5bfcae2423c8689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:06:27.467054 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-925dd32c-aad6-4f50-a44d-af104c5d7eb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:27.467669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-925dd32c-aad6-4f50-a44d-af104c5d7eb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:27.474182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1179/2378] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:06:27 2022] DELETE /identity/v3/projects/b9569a52dfd64754b5bfcae2423c8689 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:27.474523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20b378-332e-4653-8e41-13c24b3e545c service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:27.484446 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c20b378-332e-4653-8e41-13c24b3e545c service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:27.550670 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabfd28-ca6b-4d28-a43b-93f6e6bbd827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:27.551662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabfd28-ca6b-4d28-a43b-93f6e6bbd827 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:27.552028 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabfd28-ca6b-4d28-a43b-93f6e6bbd827 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:27.552399 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fabfd28-ca6b-4d28-a43b-93f6e6bbd827 admin admin] PATH_INFO: `/v3/users/dde5489066fb49a2b68b15bc62f53301` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:27.553686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fabfd28-ca6b-4d28-a43b-93f6e6bbd827 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dde5489066fb49a2b68b15bc62f53301)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:27.555744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fabfd28-ca6b-4d28-a43b-93f6e6bbd827 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:27.607553 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6fabfd28-ca6b-4d28-a43b-93f6e6bbd827 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dde5489066fb49a2b68b15bc62f53301'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:27.615977 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6fabfd28-ca6b-4d28-a43b-93f6e6bbd827 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dde5489066fb49a2b68b15bc62f53301'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:27.616702 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6fabfd28-ca6b-4d28-a43b-93f6e6bbd827 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dde5489066fb49a2b68b15bc62f53301'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:27.639268 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6fabfd28-ca6b-4d28-a43b-93f6e6bbd827 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dde5489066fb49a2b68b15bc62f53301'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:27.639844 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6fabfd28-ca6b-4d28-a43b-93f6e6bbd827 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dde5489066fb49a2b68b15bc62f53301'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:27.653855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1200/2379] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:06:27 2022] DELETE /identity/v3/users/dde5489066fb49a2b68b15bc62f53301 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:27.940656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dd32c-aad6-4f50-a44d-af104c5d7eb5 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:27.942233 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-925dd32c-aad6-4f50-a44d-af104c5d7eb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:27.956555 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925dd32c-aad6-4f50-a44d-af104c5d7eb5 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:28.013603 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ceeff-5276-48e6-bfea-4b0e0e91fb4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:28.014469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ceeff-5276-48e6-bfea-4b0e0e91fb4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:28.014787 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ceeff-5276-48e6-bfea-4b0e0e91fb4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:28.015072 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf2ceeff-5276-48e6-bfea-4b0e0e91fb4b admin admin] PATH_INFO: `/v3/projects/341857b9ea7045b2967612ab4659856f` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:28.016800 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ceeff-5276-48e6-bfea-4b0e0e91fb4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=341857b9ea7045b2967612ab4659856f)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:28.018730 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf2ceeff-5276-48e6-bfea-4b0e0e91fb4b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:28.064197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cf2ceeff-5276-48e6-bfea-4b0e0e91fb4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '341857b9ea7045b2967612ab4659856f', 'request_id': 'req-cf2ceeff-5276-48e6-bfea-4b0e0e91fb4b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:28.070671 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cf2ceeff-5276-48e6-bfea-4b0e0e91fb4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '341857b9ea7045b2967612ab4659856f', 'request_id': 'req-cf2ceeff-5276-48e6-bfea-4b0e0e91fb4b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:28.077150 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cf2ceeff-5276-48e6-bfea-4b0e0e91fb4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '341857b9ea7045b2967612ab4659856f', 'request_id': 'req-cf2ceeff-5276-48e6-bfea-4b0e0e91fb4b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:28.082794 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:28.092645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cf2ceeff-5276-48e6-bfea-4b0e0e91fb4b admin admin] The token cache is being invalidate because project 341857b9ea7045b2967612ab4659856f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:06:28.093473 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cf2ceeff-5276-48e6-bfea-4b0e0e91fb4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:28.094558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cf2ceeff-5276-48e6-bfea-4b0e0e91fb4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:28.096662 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1180/2380] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:06:27 2022] DELETE /identity/v3/projects/341857b9ea7045b2967612ab4659856f => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:28.100122 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabfd28-ca6b-4d28-a43b-93f6e6bbd827 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:28.101780 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-6fabfd28-ca6b-4d28-a43b-93f6e6bbd827 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:28.115760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fabfd28-ca6b-4d28-a43b-93f6e6bbd827 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:28.148372 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:28.174374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d66a145-936e-4f5c-b383-7a4e0fe56181 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:28.175286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66a145-936e-4f5c-b383-7a4e0fe56181 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:28.175574 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66a145-936e-4f5c-b383-7a4e0fe56181 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:28.175856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d66a145-936e-4f5c-b383-7a4e0fe56181 admin admin] PATH_INFO: `/v3/users/c769a5edd7bf42f7a7df03c1be6580da` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:28.177047 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66a145-936e-4f5c-b383-7a4e0fe56181 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c769a5edd7bf42f7a7df03c1be6580da)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:28.179009 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d66a145-936e-4f5c-b383-7a4e0fe56181 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:28.224021 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1d66a145-936e-4f5c-b383-7a4e0fe56181 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c769a5edd7bf42f7a7df03c1be6580da'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:28.231228 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1d66a145-936e-4f5c-b383-7a4e0fe56181 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c769a5edd7bf42f7a7df03c1be6580da'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:28.231949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1d66a145-936e-4f5c-b383-7a4e0fe56181 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c769a5edd7bf42f7a7df03c1be6580da'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:28.244691 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1d66a145-936e-4f5c-b383-7a4e0fe56181 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c769a5edd7bf42f7a7df03c1be6580da'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:28.247246 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-1d66a145-936e-4f5c-b383-7a4e0fe56181 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c769a5edd7bf42f7a7df03c1be6580da'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:28.263008 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1201/2381] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:06:28 2022] DELETE /identity/v3/users/c769a5edd7bf42f7a7df03c1be6580da => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:28.550926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ceeff-5276-48e6-bfea-4b0e0e91fb4b admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:28.552891 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-cf2ceeff-5276-48e6-bfea-4b0e0e91fb4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:28.574143 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf2ceeff-5276-48e6-bfea-4b0e0e91fb4b admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:28.646440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a6489a-f486-4a01-9d64-1212bacf3a34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:28.647359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6489a-f486-4a01-9d64-1212bacf3a34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:28.647641 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6489a-f486-4a01-9d64-1212bacf3a34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:28.648105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a6489a-f486-4a01-9d64-1212bacf3a34 admin admin] PATH_INFO: `/v3/projects/7aaa6b56fcdb4735b485868f9dbfe574` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:28.649306 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6489a-f486-4a01-9d64-1212bacf3a34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aaa6b56fcdb4735b485868f9dbfe574)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:28.651344 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a6489a-f486-4a01-9d64-1212bacf3a34 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:28.705963 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-35a6489a-f486-4a01-9d64-1212bacf3a34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aaa6b56fcdb4735b485868f9dbfe574', 'request_id': 'req-35a6489a-f486-4a01-9d64-1212bacf3a34'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:28.716042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-35a6489a-f486-4a01-9d64-1212bacf3a34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aaa6b56fcdb4735b485868f9dbfe574', 'request_id': 'req-35a6489a-f486-4a01-9d64-1212bacf3a34'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:28.726451 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-35a6489a-f486-4a01-9d64-1212bacf3a34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aaa6b56fcdb4735b485868f9dbfe574', 'request_id': 'req-35a6489a-f486-4a01-9d64-1212bacf3a34'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:28.756005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-35a6489a-f486-4a01-9d64-1212bacf3a34 admin admin] The token cache is being invalidate because project 7aaa6b56fcdb4735b485868f9dbfe574 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:06:28.756342 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-35a6489a-f486-4a01-9d64-1212bacf3a34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:28.757148 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-35a6489a-f486-4a01-9d64-1212bacf3a34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:06:28.758753 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1181/2382] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:06:28 2022] DELETE /identity/v3/projects/7aaa6b56fcdb4735b485868f9dbfe574 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:28.763779 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd8ee7-0b8d-4cee-ad55-7ee78fb3c0f1 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:28.764077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd8ee7-0b8d-4cee-ad55-7ee78fb3c0f1 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:28.764365 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fd8ee7-0b8d-4cee-ad55-7ee78fb3c0f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:28.784675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-30fd8ee7-0b8d-4cee-ad55-7ee78fb3c0f1 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:28.786063 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-30fd8ee7-0b8d-4cee-ad55-7ee78fb3c0f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:28.870850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1202/2383] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:28.878948 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b935f-28fc-4fe3-8c6d-14d9093f886d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:28.879351 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b935f-28fc-4fe3-8c6d-14d9093f886d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:28.879745 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b935f-28fc-4fe3-8c6d-14d9093f886d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:28.903414 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765897e8-e980-48ee-927d-79711444b029 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:28.903676 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765897e8-e980-48ee-927d-79711444b029 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:28.903967 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765897e8-e980-48ee-927d-79711444b029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:28.908002 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-ab3b935f-28fc-4fe3-8c6d-14d9093f886d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:28.911184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-ab3b935f-28fc-4fe3-8c6d-14d9093f886d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:28.926112 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-765897e8-e980-48ee-927d-79711444b029 None None] MFA Rules not processed for user `faf778e713b54a638c1db4a5db6180b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:28.927586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-765897e8-e980-48ee-927d-79711444b029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:28.928704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1182/2384] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:28.929822 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b935f-28fc-4fe3-8c6d-14d9093f886d None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:28.948959 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b935f-28fc-4fe3-8c6d-14d9093f886d None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:28.959437 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14131563-fd7e-4cf3-a04a-132a32df5582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:28.960514 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14131563-fd7e-4cf3-a04a-132a32df5582 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:28.961373 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14131563-fd7e-4cf3-a04a-132a32df5582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:28.961871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14131563-fd7e-4cf3-a04a-132a32df5582 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:28.963892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14131563-fd7e-4cf3-a04a-132a32df5582 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:06:28.964587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14131563-fd7e-4cf3-a04a-132a32df5582 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:28.969762 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14131563-fd7e-4cf3-a04a-132a32df5582 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:28.987501 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1183/2385] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:06:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 23:06:28.994339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14131563-fd7e-4cf3-a04a-132a32df5582 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:29.008241 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:29.013295 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14131563-fd7e-4cf3-a04a-132a32df5582 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:29.021359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1203/2386] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 23:06:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:29.022299 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cc568-47ff-410f-b146-952ace4c731d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:29.023480 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cc568-47ff-410f-b146-952ace4c731d admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:29.023950 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cc568-47ff-410f-b146-952ace4c731d admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:29.024392 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64cc568-47ff-410f-b146-952ace4c731d admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:29.025566 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64cc568-47ff-410f-b146-952ace4c731d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:29.028968 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64cc568-47ff-410f-b146-952ace4c731d admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:29.065479 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1184/2387] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:06:28 2022] POST /identity/v3/projects => generated 371 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:06:29.069071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765897e8-e980-48ee-927d-79711444b029 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:29.083179 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765897e8-e980-48ee-927d-79711444b029 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:29.095664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cc568-47ff-410f-b146-952ace4c731d admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:29.097854 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-e64cc568-47ff-410f-b146-952ace4c731d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:29.145671 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:INSERT=3,keystone:DELETE=8,keystone:SELECT=89 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:29.150035 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d5cbc-0b86-4651-91cb-07797a74890f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:29.150758 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d5cbc-0b86-4651-91cb-07797a74890f admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:29.151120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d5cbc-0b86-4651-91cb-07797a74890f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:29.151469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0d5cbc-0b86-4651-91cb-07797a74890f admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:29.152285 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d5cbc-0b86-4651-91cb-07797a74890f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:29.154529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0d5cbc-0b86-4651-91cb-07797a74890f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:29.179023 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1204/2388] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:06:29 2022] POST /identity/v3/users => generated 286 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:06:29.183406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d5cbc-0b86-4651-91cb-07797a74890f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:29.194093 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64cc568-47ff-410f-b146-952ace4c731d admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:29.195822 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0d5cbc-0b86-4651-91cb-07797a74890f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:29.199332 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843d3c2-40b6-4352-b6ad-83570c2da5fe service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef518c9430e4274b1066241f821f4a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:29.200001 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843d3c2-40b6-4352-b6ad-83570c2da5fe service swift] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:29.200295 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843d3c2-40b6-4352-b6ad-83570c2da5fe service swift] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:29.200583 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843d3c2-40b6-4352-b6ad-83570c2da5fe service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:29.202871 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ae834-ad62-4859-b9fa-4beffaf2eda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:29.203917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ae834-ad62-4859-b9fa-4beffaf2eda7 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:29.204221 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ae834-ad62-4859-b9fa-4beffaf2eda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:29.204503 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375ae834-ad62-4859-b9fa-4beffaf2eda7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:29.207029 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ae834-ad62-4859-b9fa-4beffaf2eda7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:29.208705 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375ae834-ad62-4859-b9fa-4beffaf2eda7 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:29.223595 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1205/2389] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:06:29.227298 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ae834-ad62-4859-b9fa-4beffaf2eda7 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:29.239586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375ae834-ad62-4859-b9fa-4beffaf2eda7 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:29.245914 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbb93e-cf72-4d1f-8608-88092d404575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:29.246785 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbb93e-cf72-4d1f-8608-88092d404575 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:29.247692 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbb93e-cf72-4d1f-8608-88092d404575 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:29.247692 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dbb93e-cf72-4d1f-8608-88092d404575 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:29.248264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dbb93e-cf72-4d1f-8608-88092d404575 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:29.250191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dbb93e-cf72-4d1f-8608-88092d404575 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:29.264654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1206/2390] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:06:29 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:06:29.272132 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbb93e-cf72-4d1f-8608-88092d404575 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:29.278079 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843d3c2-40b6-4352-b6ad-83570c2da5fe service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:29.280444 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843d3c2-40b6-4352-b6ad-83570c2da5fe service swift] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:29.283680 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dbb93e-cf72-4d1f-8608-88092d404575 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:29.289565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1185/2391] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 23:06:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 23:06:29.290437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f1323-f3f7-49c0-be52-e2b1d7c9b1c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:29.291680 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f1323-f3f7-49c0-be52-e2b1d7c9b1c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:29.291986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f1323-f3f7-49c0-be52-e2b1d7c9b1c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:29.291986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27f1323-f3f7-49c0-be52-e2b1d7c9b1c5 admin admin] PATH_INFO: `/v3/projects/91ee581f47a5419098d02102081f617d/users/61e40b7f3da047fcbea943e7f1206bcb/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:29.308603 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f1323-f3f7-49c0-be52-e2b1d7c9b1c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91ee581f47a5419098d02102081f617d, user_id=61e40b7f3da047fcbea943e7f1206bcb, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:29.311347 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27f1323-f3f7-49c0-be52-e2b1d7c9b1c5 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:29.342381 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1207/2392] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:06:29 2022] PUT /identity/v3/projects/91ee581f47a5419098d02102081f617d/users/61e40b7f3da047fcbea943e7f1206bcb/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:06:29.347912 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55492f6-0b73-4dbb-b2c1-8e4721bed9ff None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:29.348438 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55492f6-0b73-4dbb-b2c1-8e4721bed9ff None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:29.348885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55492f6-0b73-4dbb-b2c1-8e4721bed9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:29.372535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-b55492f6-0b73-4dbb-b2c1-8e4721bed9ff None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:29.374244 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-b55492f6-0b73-4dbb-b2c1-8e4721bed9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:29.464726 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1186/2393] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:29.475747 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f1323-f3f7-49c0-be52-e2b1d7c9b1c5 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:29.477554 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-d27f1323-f3f7-49c0-be52-e2b1d7c9b1c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:29.595908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27f1323-f3f7-49c0-be52-e2b1d7c9b1c5 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:29.606086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c2e72-3efa-48f5-95d1-a9c0112bec41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:29.606798 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c2e72-3efa-48f5-95d1-a9c0112bec41 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:29.607091 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c2e72-3efa-48f5-95d1-a9c0112bec41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:29.607616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94c2e72-3efa-48f5-95d1-a9c0112bec41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:29.618000 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c2e72-3efa-48f5-95d1-a9c0112bec41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:29.619910 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94c2e72-3efa-48f5-95d1-a9c0112bec41 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:29.635792 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1208/2394] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:06:30.765955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab515c-f83b-49dc-9dc8-3e9b965c6408 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:30.766520 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab515c-f83b-49dc-9dc8-3e9b965c6408 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:30.766520 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab515c-f83b-49dc-9dc8-3e9b965c6408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:30.809599 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-63ab515c-f83b-49dc-9dc8-3e9b965c6408 None None] MFA Rules not processed for user `61e40b7f3da047fcbea943e7f1206bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:30.811669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-63ab515c-f83b-49dc-9dc8-3e9b965c6408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:30.887914 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:30.893359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1187/2395] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:30 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:30.904913 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c2e72-3efa-48f5-95d1-a9c0112bec41 service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:30.907705 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-d94c2e72-3efa-48f5-95d1-a9c0112bec41 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:30.992703 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:31.009716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94c2e72-3efa-48f5-95d1-a9c0112bec41 service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:31.017512 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f731fd1-a33f-4d49-a8e2-25b5ee47949b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:31.018497 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f731fd1-a33f-4d49-a8e2-25b5ee47949b service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:31.019018 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f731fd1-a33f-4d49-a8e2-25b5ee47949b service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:31.019478 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f731fd1-a33f-4d49-a8e2-25b5ee47949b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:31.031782 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f731fd1-a33f-4d49-a8e2-25b5ee47949b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:31.034581 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f731fd1-a33f-4d49-a8e2-25b5ee47949b service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:31.051343 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1209/2396] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:32.705874 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cd9cc-afaa-4faa-95cd-7551c3a1a06a None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:32.706567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cd9cc-afaa-4faa-95cd-7551c3a1a06a None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:32.706824 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1cd9cc-afaa-4faa-95cd-7551c3a1a06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:32.730182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-6f1cd9cc-afaa-4faa-95cd-7551c3a1a06a None None] MFA Rules not processed for user `61e40b7f3da047fcbea943e7f1206bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:32.731628 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6f1cd9cc-afaa-4faa-95cd-7551c3a1a06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:32.744293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1188/2397] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:06:32 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:36.401302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f731fd1-a33f-4d49-a8e2-25b5ee47949b service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:36.403786 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-4f731fd1-a33f-4d49-a8e2-25b5ee47949b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:36.420351 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f731fd1-a33f-4d49-a8e2-25b5ee47949b service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:36.425149 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b488f513-427f-4708-85fa-7ba2972d0e4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:36.425935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488f513-427f-4708-85fa-7ba2972d0e4e service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:36.426294 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488f513-427f-4708-85fa-7ba2972d0e4e service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:36.426666 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b488f513-427f-4708-85fa-7ba2972d0e4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:36.428914 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b488f513-427f-4708-85fa-7ba2972d0e4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:36.561290 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b488f513-427f-4708-85fa-7ba2972d0e4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:36.563467 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b488f513-427f-4708-85fa-7ba2972d0e4e service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:36.576195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1210/2398] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:36 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:42.723419 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:45.449264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b30307-d8c5-47ad-b736-a484795ad70e None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:45.449916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b30307-d8c5-47ad-b736-a484795ad70e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:45.449916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b30307-d8c5-47ad-b736-a484795ad70e None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:45.451821 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1189/2399] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 23:06:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 23:06:45.458223 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b3447-a351-41a5-aceb-aecfd55dd234 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:45.458710 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b3447-a351-41a5-aceb-aecfd55dd234 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:45.458814 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b3447-a351-41a5-aceb-aecfd55dd234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:45.481064 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-ae9b3447-a351-41a5-aceb-aecfd55dd234 None None] MFA Rules not processed for user `4180f2593149437fa27effd8d1338d10`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:45.481983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ae9b3447-a351-41a5-aceb-aecfd55dd234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:45.556371 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1211/2400] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 23:06:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:06:45.568207 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b30307-d8c5-47ad-b736-a484795ad70e None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:45.569915 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f6b30307-d8c5-47ad-b736-a484795ad70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:45.628541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b30307-d8c5-47ad-b736-a484795ad70e None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:45.632070 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fde7f4-f3cc-471e-838b-5763b3cc468a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915f4296504b47d796e3a5b99550eb75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:45.632712 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fde7f4-f3cc-471e-838b-5763b3cc468a service glance] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:45.633001 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fde7f4-f3cc-471e-838b-5763b3cc468a service glance] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:45.633290 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fde7f4-f3cc-471e-838b-5763b3cc468a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:45.643846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fde7f4-f3cc-471e-838b-5763b3cc468a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:45.645263 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fde7f4-f3cc-471e-838b-5763b3cc468a service glance] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:45.655775 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1190/2401] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:45.723323 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429edd0-7d91-451b-b713-a9759c10e7c9 None None] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:45.723658 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429edd0-7d91-451b-b713-a9759c10e7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:45.723970 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f429edd0-7d91-451b-b713-a9759c10e7c9 None None] PATH_INFO: `/` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:45.725413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1212/2402] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 23:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 23:06:45.730656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c0817-17e4-4bf6-ac76-28dbb32c901b None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:45.730949 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c0817-17e4-4bf6-ac76-28dbb32c901b None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:45.731227 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c0817-17e4-4bf6-ac76-28dbb32c901b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:45.748935 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-bc1c0817-17e4-4bf6-ac76-28dbb32c901b None None] MFA Rules not processed for user `6b794f7aa2a7488e91f7d9ea35a45d78`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:45.750134 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-bc1c0817-17e4-4bf6-ac76-28dbb32c901b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:45.757395 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1191/2403] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 23:06:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:45.769365 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429edd0-7d91-451b-b713-a9759c10e7c9 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:45.780315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f429edd0-7d91-451b-b713-a9759c10e7c9 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:45.783967 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6efe1595-889e-449f-8575-868d7e71d933 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:45.784613 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe1595-889e-449f-8575-868d7e71d933 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:45.784957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe1595-889e-449f-8575-868d7e71d933 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:45.785239 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6efe1595-889e-449f-8575-868d7e71d933 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:45.792911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe1595-889e-449f-8575-868d7e71d933 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:45.794336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6efe1595-889e-449f-8575-868d7e71d933 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:45.802725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1213/2404] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:45.892675 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b276c32-d156-468a-9ed4-27b450ee9f46 None None] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:45.892969 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b276c32-d156-468a-9ed4-27b450ee9f46 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:45.893247 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b276c32-d156-468a-9ed4-27b450ee9f46 None None] PATH_INFO: `/` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:45.894684 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1192/2405] 10.222.0.17 () {54 vars in 921 bytes} [Thu Jun 16 23:06:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 23:06:45.899933 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46c29b-824f-4d0a-a65a-0b9631055666 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:45.900228 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46c29b-824f-4d0a-a65a-0b9631055666 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:45.900513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a46c29b-824f-4d0a-a65a-0b9631055666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:45.918720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-1a46c29b-824f-4d0a-a65a-0b9631055666 None None] MFA Rules not processed for user `4180f2593149437fa27effd8d1338d10`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:45.920473 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-1a46c29b-824f-4d0a-a65a-0b9631055666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:45.929887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1214/2406] 10.222.0.17 () {58 vars in 1018 bytes} [Thu Jun 16 23:06:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:45.941576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b276c32-d156-468a-9ed4-27b450ee9f46 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:45.953189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b276c32-d156-468a-9ed4-27b450ee9f46 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:45.957216 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50974ecf-ae6e-48e4-9078-72de0a990c91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:45.957931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50974ecf-ae6e-48e4-9078-72de0a990c91 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:45.958285 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50974ecf-ae6e-48e4-9078-72de0a990c91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:45.958657 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50974ecf-ae6e-48e4-9078-72de0a990c91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:45.970074 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50974ecf-ae6e-48e4-9078-72de0a990c91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:45.971534 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50974ecf-ae6e-48e4-9078-72de0a990c91 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:45.987037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1193/2407] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:06:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:06:49.461946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8610da-aabb-4f1c-9dc9-e331828cd127 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:49.462436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8610da-aabb-4f1c-9dc9-e331828cd127 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:49.462553 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8610da-aabb-4f1c-9dc9-e331828cd127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:49.480240 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-df8610da-aabb-4f1c-9dc9-e331828cd127 None None] MFA Rules not processed for user `faf778e713b54a638c1db4a5db6180b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:06:49.481423 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-df8610da-aabb-4f1c-9dc9-e331828cd127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:06:49.527399 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:06:49.535955 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1215/2408] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 23:06:49 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:06:49.548242 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50974ecf-ae6e-48e4-9078-72de0a990c91 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:06:49.612642 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50974ecf-ae6e-48e4-9078-72de0a990c91 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:06:49.617976 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373d046a-959e-4649-93c3-978f6e2370ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef518c9430e4274b1066241f821f4a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:06:49.618635 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d046a-959e-4649-93c3-978f6e2370ac service swift] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:06:49.618923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d046a-959e-4649-93c3-978f6e2370ac service swift] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:06:49.619199 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373d046a-959e-4649-93c3-978f6e2370ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:06:49.627108 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d046a-959e-4649-93c3-978f6e2370ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:06:49.628682 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-373d046a-959e-4649-93c3-978f6e2370ac service swift] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:06:49.636803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1194/2409] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 23:06:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 23:06:59.633632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=67 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:00.272703 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa0b06-687a-4aed-aa8a-2eaf836f5dc6 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:00.273214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa0b06-687a-4aed-aa8a-2eaf836f5dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:00.273329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa0b06-687a-4aed-aa8a-2eaf836f5dc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:00.290430 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a0aa0b06-687a-4aed-aa8a-2eaf836f5dc6 None None] MFA Rules not processed for user `61e40b7f3da047fcbea943e7f1206bcb`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:00.291723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a0aa0b06-687a-4aed-aa8a-2eaf836f5dc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:00.298961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1216/2410] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:06.292786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7d5b3-1fe5-48c5-b41e-223884682538 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:06.293226 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7d5b3-1fe5-48c5-b41e-223884682538 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:06.293345 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc7d5b3-1fe5-48c5-b41e-223884682538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:06.312737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-dfc7d5b3-1fe5-48c5-b41e-223884682538 None None] MFA Rules not processed for user `faf778e713b54a638c1db4a5db6180b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:06.314201 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-dfc7d5b3-1fe5-48c5-b41e-223884682538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:06.321398 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1195/2411] 10.222.0.17 () {58 vars in 967 bytes} [Thu Jun 16 23:07:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:06.334081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa0b06-687a-4aed-aa8a-2eaf836f5dc6 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:06.346408 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa0b06-687a-4aed-aa8a-2eaf836f5dc6 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:06.351446 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3748a8f-2f8c-4c96-b0cf-e41eb2c535fb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef518c9430e4274b1066241f821f4a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:06.352092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3748a8f-2f8c-4c96-b0cf-e41eb2c535fb service swift] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:06.352383 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3748a8f-2f8c-4c96-b0cf-e41eb2c535fb service swift] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:06.352666 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3748a8f-2f8c-4c96-b0cf-e41eb2c535fb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:06.360734 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3748a8f-2f8c-4c96-b0cf-e41eb2c535fb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:06.362338 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3748a8f-2f8c-4c96-b0cf-e41eb2c535fb service swift] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:06.370362 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1217/2412] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 23:07:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 23:07:16.306667 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:16.367117 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=9 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:19.977539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7d5b3-1fe5-48c5-b41e-223884682538 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:19.987556 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc7d5b3-1fe5-48c5-b41e-223884682538 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:19.991124 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16996427-5298-48e6-a604-bedf933dadac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:19.991768 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16996427-5298-48e6-a604-bedf933dadac service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:19.992058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16996427-5298-48e6-a604-bedf933dadac service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:19.992339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16996427-5298-48e6-a604-bedf933dadac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:19.994114 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-16996427-5298-48e6-a604-bedf933dadac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:20.052302 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16996427-5298-48e6-a604-bedf933dadac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:20.053727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16996427-5298-48e6-a604-bedf933dadac service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:20.062617 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1196/2413] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:07:22.495473 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b268614-ed33-40bc-87f4-4ec9012ee279 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:22.495922 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b268614-ed33-40bc-87f4-4ec9012ee279 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:22.496028 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b268614-ed33-40bc-87f4-4ec9012ee279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:22.514583 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-0b268614-ed33-40bc-87f4-4ec9012ee279 None None] MFA Rules not processed for user `faf778e713b54a638c1db4a5db6180b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:22.515806 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-0b268614-ed33-40bc-87f4-4ec9012ee279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:22.522957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1218/2414] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 23:07:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:22.534967 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16996427-5298-48e6-a604-bedf933dadac service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:22.546470 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16996427-5298-48e6-a604-bedf933dadac service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:22.551530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc19798-2cb1-46a5-acdf-884ec09d3dd2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef518c9430e4274b1066241f821f4a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:22.552172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc19798-2cb1-46a5-acdf-884ec09d3dd2 service swift] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:22.552461 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc19798-2cb1-46a5-acdf-884ec09d3dd2 service swift] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:22.552750 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc19798-2cb1-46a5-acdf-884ec09d3dd2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:22.561291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc19798-2cb1-46a5-acdf-884ec09d3dd2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:22.562892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc19798-2cb1-46a5-acdf-884ec09d3dd2 service swift] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:22.571095 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1197/2415] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 23:07:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 23:07:27.237896 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b268614-ed33-40bc-87f4-4ec9012ee279 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:27.249945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b268614-ed33-40bc-87f4-4ec9012ee279 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:27.254679 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095f34b-cf86-4346-a4de-af587ef7535c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:27.255924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095f34b-cf86-4346-a4de-af587ef7535c service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:27.256462 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095f34b-cf86-4346-a4de-af587ef7535c service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:27.256947 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f095f34b-cf86-4346-a4de-af587ef7535c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:27.260635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-f095f34b-cf86-4346-a4de-af587ef7535c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:27.280548 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095f34b-cf86-4346-a4de-af587ef7535c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:27.284438 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f095f34b-cf86-4346-a4de-af587ef7535c service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:27.322686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1219/2416] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:07:32.159716 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc19798-2cb1-46a5-acdf-884ec09d3dd2 service swift] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:32.161243 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-ebc19798-2cb1-46a5-acdf-884ec09d3dd2 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:32.384367 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc19798-2cb1-46a5-acdf-884ec09d3dd2 service swift] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:32.405779 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe471f-7d5a-4c96-a899-ddd3a6d3d042 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9593939641c4c7797c1981206b40e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:32.406476 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe471f-7d5a-4c96-a899-ddd3a6d3d042 service placement] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:32.406803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe471f-7d5a-4c96-a899-ddd3a6d3d042 service placement] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:32.407447 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe471f-7d5a-4c96-a899-ddd3a6d3d042 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:32.409523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-9efe471f-7d5a-4c96-a899-ddd3a6d3d042 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:32.423424 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe471f-7d5a-4c96-a899-ddd3a6d3d042 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:32.438550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe471f-7d5a-4c96-a899-ddd3a6d3d042 service placement] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:32.452629 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1198/2417] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:07:37.299264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=9 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:37.992047 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095f34b-cf86-4346-a4de-af587ef7535c service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:38.016145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f095f34b-cf86-4346-a4de-af587ef7535c service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:38.022558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a935854-9159-40b6-b5c9-ebefab76edd0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9593939641c4c7797c1981206b40e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:38.023605 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a935854-9159-40b6-b5c9-ebefab76edd0 service placement] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:38.024119 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a935854-9159-40b6-b5c9-ebefab76edd0 service placement] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:38.024600 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a935854-9159-40b6-b5c9-ebefab76edd0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:38.027394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-8a935854-9159-40b6-b5c9-ebefab76edd0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:38.045946 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a935854-9159-40b6-b5c9-ebefab76edd0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:38.048345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a935854-9159-40b6-b5c9-ebefab76edd0 service placement] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:38.064409 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1220/2418] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:07:38.069469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe471f-7d5a-4c96-a899-ddd3a6d3d042 service placement] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:38.071493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-9efe471f-7d5a-4c96-a899-ddd3a6d3d042 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:38.094821 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe471f-7d5a-4c96-a899-ddd3a6d3d042 service placement] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:38.100188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839470cb-c305-4320-b3b6-3a598d98b7bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:38.101137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839470cb-c305-4320-b3b6-3a598d98b7bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:38.101493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839470cb-c305-4320-b3b6-3a598d98b7bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:38.101839 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839470cb-c305-4320-b3b6-3a598d98b7bd admin admin] PATH_INFO: `/v3/users/01dd0e5661f44d05a01d7a5e82dbadf3` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:38.103709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839470cb-c305-4320-b3b6-3a598d98b7bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=01dd0e5661f44d05a01d7a5e82dbadf3)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:38.106412 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839470cb-c305-4320-b3b6-3a598d98b7bd admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:38.165202 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-839470cb-c305-4320-b3b6-3a598d98b7bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01dd0e5661f44d05a01d7a5e82dbadf3'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:38.166242 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-839470cb-c305-4320-b3b6-3a598d98b7bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01dd0e5661f44d05a01d7a5e82dbadf3'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:38.176619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-839470cb-c305-4320-b3b6-3a598d98b7bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01dd0e5661f44d05a01d7a5e82dbadf3'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:38.177425 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-839470cb-c305-4320-b3b6-3a598d98b7bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01dd0e5661f44d05a01d7a5e82dbadf3'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:38.188803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:38.193204 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-839470cb-c305-4320-b3b6-3a598d98b7bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01dd0e5661f44d05a01d7a5e82dbadf3'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:38.210402 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1199/2419] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:07:38 2022] DELETE /identity/v3/users/01dd0e5661f44d05a01d7a5e82dbadf3 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:38.445564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a935854-9159-40b6-b5c9-ebefab76edd0 service placement] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:38.459921 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a935854-9159-40b6-b5c9-ebefab76edd0 service placement] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:38.551629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b984af-cab5-46d1-8d3d-dcebc6915480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:38.552695 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b984af-cab5-46d1-8d3d-dcebc6915480 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:38.553141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b984af-cab5-46d1-8d3d-dcebc6915480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:38.553584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b984af-cab5-46d1-8d3d-dcebc6915480 admin admin] PATH_INFO: `/v3/projects/b30b90ec4d5e45468a4fa0fb98e46393` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:38.555225 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b984af-cab5-46d1-8d3d-dcebc6915480 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b30b90ec4d5e45468a4fa0fb98e46393)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:38.559071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b984af-cab5-46d1-8d3d-dcebc6915480 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:38.643774 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a7b984af-cab5-46d1-8d3d-dcebc6915480 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b30b90ec4d5e45468a4fa0fb98e46393', 'request_id': 'req-a7b984af-cab5-46d1-8d3d-dcebc6915480'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:38.655414 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a7b984af-cab5-46d1-8d3d-dcebc6915480 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b30b90ec4d5e45468a4fa0fb98e46393', 'request_id': 'req-a7b984af-cab5-46d1-8d3d-dcebc6915480'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:38.664111 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a7b984af-cab5-46d1-8d3d-dcebc6915480 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b30b90ec4d5e45468a4fa0fb98e46393', 'request_id': 'req-a7b984af-cab5-46d1-8d3d-dcebc6915480'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:38.679907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a7b984af-cab5-46d1-8d3d-dcebc6915480 admin admin] The token cache is being invalidate because project b30b90ec4d5e45468a4fa0fb98e46393 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:07:38.680198 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a7b984af-cab5-46d1-8d3d-dcebc6915480 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:38.681034 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a7b984af-cab5-46d1-8d3d-dcebc6915480 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:38.683523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1221/2420] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:07:38 2022] DELETE /identity/v3/projects/b30b90ec4d5e45468a4fa0fb98e46393 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:07:38.687530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839470cb-c305-4320-b3b6-3a598d98b7bd admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:38.688951 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-839470cb-c305-4320-b3b6-3a598d98b7bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:38.704890 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839470cb-c305-4320-b3b6-3a598d98b7bd admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:38.780693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66c9a9-e2e6-401c-ada1-bcdad51d23b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:38.782006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66c9a9-e2e6-401c-ada1-bcdad51d23b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:38.782299 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66c9a9-e2e6-401c-ada1-bcdad51d23b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:38.782585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f66c9a9-e2e6-401c-ada1-bcdad51d23b4 admin admin] PATH_INFO: `/v3/users/105edf55cdfc4085bde1ccfffddd325e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:38.783924 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66c9a9-e2e6-401c-ada1-bcdad51d23b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=105edf55cdfc4085bde1ccfffddd325e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:38.786284 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f66c9a9-e2e6-401c-ada1-bcdad51d23b4 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:38.855561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-9f66c9a9-e2e6-401c-ada1-bcdad51d23b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '105edf55cdfc4085bde1ccfffddd325e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:38.856544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-9f66c9a9-e2e6-401c-ada1-bcdad51d23b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '105edf55cdfc4085bde1ccfffddd325e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:38.867318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-9f66c9a9-e2e6-401c-ada1-bcdad51d23b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '105edf55cdfc4085bde1ccfffddd325e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:38.869029 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-9f66c9a9-e2e6-401c-ada1-bcdad51d23b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '105edf55cdfc4085bde1ccfffddd325e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:38.891970 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-9f66c9a9-e2e6-401c-ada1-bcdad51d23b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '105edf55cdfc4085bde1ccfffddd325e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:38.913938 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1200/2421] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:07:38 2022] DELETE /identity/v3/users/105edf55cdfc4085bde1ccfffddd325e => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:07:39.168335 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b984af-cab5-46d1-8d3d-dcebc6915480 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:39.170783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a7b984af-cab5-46d1-8d3d-dcebc6915480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:39.210526 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b984af-cab5-46d1-8d3d-dcebc6915480 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:39.310732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cf4e0-a261-43d8-bfd1-68aa22257690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:39.311864 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cf4e0-a261-43d8-bfd1-68aa22257690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:39.312215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cf4e0-a261-43d8-bfd1-68aa22257690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:39.312599 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69cf4e0-a261-43d8-bfd1-68aa22257690 admin admin] PATH_INFO: `/v3/projects/1c9163c4876f4500883459a1afca5edd` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:39.314190 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69cf4e0-a261-43d8-bfd1-68aa22257690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c9163c4876f4500883459a1afca5edd)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:39.316288 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69cf4e0-a261-43d8-bfd1-68aa22257690 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:39.340135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:39.373679 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a69cf4e0-a261-43d8-bfd1-68aa22257690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c9163c4876f4500883459a1afca5edd', 'request_id': 'req-a69cf4e0-a261-43d8-bfd1-68aa22257690'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:39.381496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a69cf4e0-a261-43d8-bfd1-68aa22257690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c9163c4876f4500883459a1afca5edd', 'request_id': 'req-a69cf4e0-a261-43d8-bfd1-68aa22257690'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:39.389098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a69cf4e0-a261-43d8-bfd1-68aa22257690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c9163c4876f4500883459a1afca5edd', 'request_id': 'req-a69cf4e0-a261-43d8-bfd1-68aa22257690'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:39.404273 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a69cf4e0-a261-43d8-bfd1-68aa22257690 admin admin] The token cache is being invalidate because project 1c9163c4876f4500883459a1afca5edd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:07:39.404596 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a69cf4e0-a261-43d8-bfd1-68aa22257690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:39.405354 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a69cf4e0-a261-43d8-bfd1-68aa22257690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:39.407166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1222/2422] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:07:39 2022] DELETE /identity/v3/projects/1c9163c4876f4500883459a1afca5edd => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:39.765997 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66c9a9-e2e6-401c-ada1-bcdad51d23b4 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:39.767616 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-9f66c9a9-e2e6-401c-ada1-bcdad51d23b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:39.833851 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f66c9a9-e2e6-401c-ada1-bcdad51d23b4 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:39.853270 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:39.894537 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2994c6d4-7481-4464-abf0-6ef73053f76c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:39.895585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994c6d4-7481-4464-abf0-6ef73053f76c service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:39.895959 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994c6d4-7481-4464-abf0-6ef73053f76c service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:39.896299 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2994c6d4-7481-4464-abf0-6ef73053f76c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:39.898331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-2994c6d4-7481-4464-abf0-6ef73053f76c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:40.002524 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994c6d4-7481-4464-abf0-6ef73053f76c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:40.004010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2994c6d4-7481-4464-abf0-6ef73053f76c service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:40.027115 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1201/2423] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:07:40.430002 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cf4e0-a261-43d8-bfd1-68aa22257690 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:40.431763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a69cf4e0-a261-43d8-bfd1-68aa22257690 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:40.446945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69cf4e0-a261-43d8-bfd1-68aa22257690 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:40.455807 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54faf746-90a4-4558-9ef8-da0dc1870c5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:40.456243 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54faf746-90a4-4558-9ef8-da0dc1870c5d service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:40.456532 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54faf746-90a4-4558-9ef8-da0dc1870c5d service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:40.456815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54faf746-90a4-4558-9ef8-da0dc1870c5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:40.460259 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-54faf746-90a4-4558-9ef8-da0dc1870c5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:40.483144 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54faf746-90a4-4558-9ef8-da0dc1870c5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:40.484603 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54faf746-90a4-4558-9ef8-da0dc1870c5d service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:40.496153 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1223/2424] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:07:42.464775 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422ec89-c0f3-4441-b458-cb0eed863de0 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:42.465296 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422ec89-c0f3-4441-b458-cb0eed863de0 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:42.465393 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0422ec89-c0f3-4441-b458-cb0eed863de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:42.485885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-0422ec89-c0f3-4441-b458-cb0eed863de0 None None] MFA Rules not processed for user `faf778e713b54a638c1db4a5db6180b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:42.487270 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-0422ec89-c0f3-4441-b458-cb0eed863de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:42.581407 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1202/2425] 10.222.0.17 () {58 vars in 967 bytes} [Thu Jun 16 23:07:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:42.602027 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54faf746-90a4-4558-9ef8-da0dc1870c5d service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:42.605128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-54faf746-90a4-4558-9ef8-da0dc1870c5d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:42.739147 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54faf746-90a4-4558-9ef8-da0dc1870c5d service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:42.746004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47262c-39d3-436e-b7b9-8080fdd3fe92 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef518c9430e4274b1066241f821f4a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:42.746957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47262c-39d3-436e-b7b9-8080fdd3fe92 service swift] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:42.747406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47262c-39d3-436e-b7b9-8080fdd3fe92 service swift] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:42.747629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f47262c-39d3-436e-b7b9-8080fdd3fe92 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:42.757569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47262c-39d3-436e-b7b9-8080fdd3fe92 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:42.760079 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f47262c-39d3-436e-b7b9-8080fdd3fe92 service swift] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:42.774557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1224/2426] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 23:07:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 23:07:44.064664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422ec89-c0f3-4441-b458-cb0eed863de0 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:44.066162 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-0422ec89-c0f3-4441-b458-cb0eed863de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:44.095981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0422ec89-c0f3-4441-b458-cb0eed863de0 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:44.101205 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-708ed51f-0017-41e5-9f7c-f6dc3b43368c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:44.102344 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ed51f-0017-41e5-9f7c-f6dc3b43368c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:44.102660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ed51f-0017-41e5-9f7c-f6dc3b43368c admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:44.102946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-708ed51f-0017-41e5-9f7c-f6dc3b43368c admin admin] PATH_INFO: `/v3/users/6a993f5d14b64fccad723abecfcadf90` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:44.104221 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ed51f-0017-41e5-9f7c-f6dc3b43368c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a993f5d14b64fccad723abecfcadf90)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:44.107575 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-708ed51f-0017-41e5-9f7c-f6dc3b43368c admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:44.200013 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-708ed51f-0017-41e5-9f7c-f6dc3b43368c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a993f5d14b64fccad723abecfcadf90'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:44.200683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-708ed51f-0017-41e5-9f7c-f6dc3b43368c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a993f5d14b64fccad723abecfcadf90'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:44.212809 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-708ed51f-0017-41e5-9f7c-f6dc3b43368c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a993f5d14b64fccad723abecfcadf90'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:44.214228 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-708ed51f-0017-41e5-9f7c-f6dc3b43368c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a993f5d14b64fccad723abecfcadf90'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:44.232944 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:44.234372 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-708ed51f-0017-41e5-9f7c-f6dc3b43368c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a993f5d14b64fccad723abecfcadf90'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:44.255147 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1203/2427] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:07:44 2022] DELETE /identity/v3/users/6a993f5d14b64fccad723abecfcadf90 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:44.525747 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47262c-39d3-436e-b7b9-8080fdd3fe92 service swift] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:44.540639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f47262c-39d3-436e-b7b9-8080fdd3fe92 service swift] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:44.631322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e1db6f-dcc2-4f6b-897c-bf404f48d0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:44.632476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1db6f-dcc2-4f6b-897c-bf404f48d0d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:44.633090 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1db6f-dcc2-4f6b-897c-bf404f48d0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:44.633529 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e1db6f-dcc2-4f6b-897c-bf404f48d0d4 admin admin] PATH_INFO: `/v3/projects/f3b1be8431b0434d8bd6abca293127d5` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:44.635117 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1db6f-dcc2-4f6b-897c-bf404f48d0d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3b1be8431b0434d8bd6abca293127d5)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:44.637703 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e1db6f-dcc2-4f6b-897c-bf404f48d0d4 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:44.683009 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:44.726260 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a6e1db6f-dcc2-4f6b-897c-bf404f48d0d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3b1be8431b0434d8bd6abca293127d5', 'request_id': 'req-a6e1db6f-dcc2-4f6b-897c-bf404f48d0d4'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:44.738013 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a6e1db6f-dcc2-4f6b-897c-bf404f48d0d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3b1be8431b0434d8bd6abca293127d5', 'request_id': 'req-a6e1db6f-dcc2-4f6b-897c-bf404f48d0d4'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:44.747823 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a6e1db6f-dcc2-4f6b-897c-bf404f48d0d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3b1be8431b0434d8bd6abca293127d5', 'request_id': 'req-a6e1db6f-dcc2-4f6b-897c-bf404f48d0d4'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:44.768592 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a6e1db6f-dcc2-4f6b-897c-bf404f48d0d4 admin admin] The token cache is being invalidate because project f3b1be8431b0434d8bd6abca293127d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:07:44.768592 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a6e1db6f-dcc2-4f6b-897c-bf404f48d0d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:44.768592 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a6e1db6f-dcc2-4f6b-897c-bf404f48d0d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:44.771011 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1225/2428] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:07:44 2022] DELETE /identity/v3/projects/f3b1be8431b0434d8bd6abca293127d5 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:44.776386 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068dc66-1c4c-45f5-a0a1-f9853ada858f None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:44.776979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068dc66-1c4c-45f5-a0a1-f9853ada858f None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:44.777473 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068dc66-1c4c-45f5-a0a1-f9853ada858f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:44.830505 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-7068dc66-1c4c-45f5-a0a1-f9853ada858f None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:44.831922 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-7068dc66-1c4c-45f5-a0a1-f9853ada858f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:44.922969 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1204/2429] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:44.931400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8bb4ce-fcbd-426b-950b-6f4d5afcb99b None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:44.931757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8bb4ce-fcbd-426b-950b-6f4d5afcb99b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:44.932164 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8bb4ce-fcbd-426b-950b-6f4d5afcb99b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:44.954559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-dc8bb4ce-fcbd-426b-950b-6f4d5afcb99b None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:44.955898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-dc8bb4ce-fcbd-426b-950b-6f4d5afcb99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:44.968535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1226/2430] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:44.969870 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068dc66-1c4c-45f5-a0a1-f9853ada858f None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:44.985232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7068dc66-1c4c-45f5-a0a1-f9853ada858f None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:44.987042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8bb4ce-fcbd-426b-950b-6f4d5afcb99b None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:44.988736 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-dc8bb4ce-fcbd-426b-950b-6f4d5afcb99b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:45.002450 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfa193-bb7f-4bff-b46d-3540f5bb2e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:45.003065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfa193-bb7f-4bff-b46d-3540f5bb2e3a admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:45.003394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfa193-bb7f-4bff-b46d-3540f5bb2e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:45.003680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bfa193-bb7f-4bff-b46d-3540f5bb2e3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:45.004563 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfa193-bb7f-4bff-b46d-3540f5bb2e3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:07:45.005122 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfa193-bb7f-4bff-b46d-3540f5bb2e3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:45.007255 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bfa193-bb7f-4bff-b46d-3540f5bb2e3a admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:45.013050 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8bb4ce-fcbd-426b-950b-6f4d5afcb99b None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:45.024352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e995b-210f-48a3-9277-5302c68dfe64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:45.029536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e995b-210f-48a3-9277-5302c68dfe64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:45.030031 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e995b-210f-48a3-9277-5302c68dfe64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:45.031212 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5e995b-210f-48a3-9277-5302c68dfe64 admin admin] PATH_INFO: `/v3/users/9e3a2d16133b4af8a208682de8a7d111` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:45.038152 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e995b-210f-48a3-9277-5302c68dfe64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9e3a2d16133b4af8a208682de8a7d111)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:45.044606 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1205/2431] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:07:45.044765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5e995b-210f-48a3-9277-5302c68dfe64 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:45.046470 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfa193-bb7f-4bff-b46d-3540f5bb2e3a admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:45.066698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bfa193-bb7f-4bff-b46d-3540f5bb2e3a admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:45.072140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc2431-1842-4913-84a3-698a6b225186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:45.073655 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc2431-1842-4913-84a3-698a6b225186 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:45.074252 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc2431-1842-4913-84a3-698a6b225186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:45.074760 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cc2431-1842-4913-84a3-698a6b225186 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:45.075805 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc2431-1842-4913-84a3-698a6b225186 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:45.078038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18cc2431-1842-4913-84a3-698a6b225186 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:45.117417 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5a5e995b-210f-48a3-9277-5302c68dfe64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9e3a2d16133b4af8a208682de8a7d111'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.126337 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5a5e995b-210f-48a3-9277-5302c68dfe64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9e3a2d16133b4af8a208682de8a7d111'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.131650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5a5e995b-210f-48a3-9277-5302c68dfe64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9e3a2d16133b4af8a208682de8a7d111'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.147126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1206/2432] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:07:45 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:07:45.149481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5a5e995b-210f-48a3-9277-5302c68dfe64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9e3a2d16133b4af8a208682de8a7d111'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.150459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5a5e995b-210f-48a3-9277-5302c68dfe64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9e3a2d16133b4af8a208682de8a7d111'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.154495 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc2431-1842-4913-84a3-698a6b225186 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:45.158245 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-18cc2431-1842-4913-84a3-698a6b225186 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:45.178508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1227/2433] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:07:44 2022] DELETE /identity/v3/users/9e3a2d16133b4af8a208682de8a7d111 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:45.182720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e995b-210f-48a3-9277-5302c68dfe64 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:45.187911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5a5e995b-210f-48a3-9277-5302c68dfe64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:45.251902 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cc2431-1842-4913-84a3-698a6b225186 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:45.310008 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5e995b-210f-48a3-9277-5302c68dfe64 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:45.315860 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992946d5-d262-4ef7-9ea1-d1530e05dcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:45.316583 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992946d5-d262-4ef7-9ea1-d1530e05dcb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:45.316890 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992946d5-d262-4ef7-9ea1-d1530e05dcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:45.317480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992946d5-d262-4ef7-9ea1-d1530e05dcb0 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:45.318429 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992946d5-d262-4ef7-9ea1-d1530e05dcb0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:45.320524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992946d5-d262-4ef7-9ea1-d1530e05dcb0 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:45.346716 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:45.356344 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1228/2434] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:07:45 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:07:45.360531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992946d5-d262-4ef7-9ea1-d1530e05dcb0 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:45.362673 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482e712-b751-4859-881f-98b768fc194b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:45.363823 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482e712-b751-4859-881f-98b768fc194b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:45.364272 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482e712-b751-4859-881f-98b768fc194b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:45.364834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c482e712-b751-4859-881f-98b768fc194b admin admin] PATH_INFO: `/v3/users/61e40b7f3da047fcbea943e7f1206bcb` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:45.367709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482e712-b751-4859-881f-98b768fc194b admin admin] RBAC: Authorizing `identity:delete_user(user_id=61e40b7f3da047fcbea943e7f1206bcb)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:45.370795 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992946d5-d262-4ef7-9ea1-d1530e05dcb0 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:45.371250 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c482e712-b751-4859-881f-98b768fc194b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:45.375879 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0d1a9-054b-4480-9c86-7120e8ef061e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:45.376586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0d1a9-054b-4480-9c86-7120e8ef061e admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:45.376880 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0d1a9-054b-4480-9c86-7120e8ef061e admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:45.377163 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda0d1a9-054b-4480-9c86-7120e8ef061e admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:45.377898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0d1a9-054b-4480-9c86-7120e8ef061e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:45.380833 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda0d1a9-054b-4480-9c86-7120e8ef061e admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:45.392301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:DELETE=16,keystone:SELECT=79,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:45.395909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1229/2435] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:07:45.397242 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0d1a9-054b-4480-9c86-7120e8ef061e admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:45.418471 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda0d1a9-054b-4480-9c86-7120e8ef061e admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:45.433566 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc152a2-4a1e-422d-b966-a7e23619c149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:45.434438 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc152a2-4a1e-422d-b966-a7e23619c149 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:45.435487 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc152a2-4a1e-422d-b966-a7e23619c149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:45.435914 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc152a2-4a1e-422d-b966-a7e23619c149 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:45.437289 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc152a2-4a1e-422d-b966-a7e23619c149 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:45.446486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cc152a2-4a1e-422d-b966-a7e23619c149 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:45.453796 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c482e712-b751-4859-881f-98b768fc194b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61e40b7f3da047fcbea943e7f1206bcb'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.454699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c482e712-b751-4859-881f-98b768fc194b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61e40b7f3da047fcbea943e7f1206bcb'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.467374 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c482e712-b751-4859-881f-98b768fc194b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61e40b7f3da047fcbea943e7f1206bcb'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.468283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c482e712-b751-4859-881f-98b768fc194b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61e40b7f3da047fcbea943e7f1206bcb'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.473544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1230/2436] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:07:45.474953 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc152a2-4a1e-422d-b966-a7e23619c149 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:45.476747 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5cc152a2-4a1e-422d-b966-a7e23619c149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:45.483709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c482e712-b751-4859-881f-98b768fc194b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61e40b7f3da047fcbea943e7f1206bcb'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.492295 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cc152a2-4a1e-422d-b966-a7e23619c149 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:45.497903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fa615-abce-4844-b4b0-5ca25792e1a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:45.498880 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fa615-abce-4844-b4b0-5ca25792e1a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:45.499303 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fa615-abce-4844-b4b0-5ca25792e1a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:45.499703 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fa615-abce-4844-b4b0-5ca25792e1a1 admin admin] PATH_INFO: `/v3/projects/0526c12e8bc947209e24daf5ddfad7be` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:45.500485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1207/2437] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:07:45 2022] DELETE /identity/v3/users/61e40b7f3da047fcbea943e7f1206bcb => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:45.501602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fa615-abce-4844-b4b0-5ca25792e1a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0526c12e8bc947209e24daf5ddfad7be)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:45.503524 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482e712-b751-4859-881f-98b768fc194b admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:45.505974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c482e712-b751-4859-881f-98b768fc194b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:45.507860 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fa615-abce-4844-b4b0-5ca25792e1a1 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:45.567395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9f2fa615-abce-4844-b4b0-5ca25792e1a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0526c12e8bc947209e24daf5ddfad7be', 'request_id': 'req-9f2fa615-abce-4844-b4b0-5ca25792e1a1'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.577510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9f2fa615-abce-4844-b4b0-5ca25792e1a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0526c12e8bc947209e24daf5ddfad7be', 'request_id': 'req-9f2fa615-abce-4844-b4b0-5ca25792e1a1'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.584943 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9f2fa615-abce-4844-b4b0-5ca25792e1a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0526c12e8bc947209e24daf5ddfad7be', 'request_id': 'req-9f2fa615-abce-4844-b4b0-5ca25792e1a1'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.590739 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c482e712-b751-4859-881f-98b768fc194b admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:45.602830 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9f2fa615-abce-4844-b4b0-5ca25792e1a1 admin admin] The token cache is being invalidate because project 0526c12e8bc947209e24daf5ddfad7be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:07:45.603248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9f2fa615-abce-4844-b4b0-5ca25792e1a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.604437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-9f2fa615-abce-4844-b4b0-5ca25792e1a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.606332 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1231/2438] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:07:45 2022] DELETE /identity/v3/projects/0526c12e8bc947209e24daf5ddfad7be => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:45.611317 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0ee27-b6c5-4b56-b5a0-81e8006b6cee None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:45.611645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0ee27-b6c5-4b56-b5a0-81e8006b6cee None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:45.612071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac0ee27-b6c5-4b56-b5a0-81e8006b6cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:45.631964 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-eac0ee27-b6c5-4b56-b5a0-81e8006b6cee None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:45.633213 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-eac0ee27-b6c5-4b56-b5a0-81e8006b6cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:45.653495 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a60b4-e08c-40b9-96c5-d567bb78c089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:45.654694 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a60b4-e08c-40b9-96c5-d567bb78c089 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:45.655055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a60b4-e08c-40b9-96c5-d567bb78c089 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:45.655407 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a60b4-e08c-40b9-96c5-d567bb78c089 admin admin] PATH_INFO: `/v3/projects/eb2bd01b77f948ceb4ac08cc658d39fb/users/19d1751261ee4972add64c8e71a6bc47/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:45.671884 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a60b4-e08c-40b9-96c5-d567bb78c089 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb2bd01b77f948ceb4ac08cc658d39fb, user_id=19d1751261ee4972add64c8e71a6bc47, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:45.674746 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a60b4-e08c-40b9-96c5-d567bb78c089 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:45.692198 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1208/2439] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:07:45 2022] PUT /identity/v3/projects/eb2bd01b77f948ceb4ac08cc658d39fb/users/19d1751261ee4972add64c8e71a6bc47/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:45.695615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a60b4-e08c-40b9-96c5-d567bb78c089 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:45.697234 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d20a60b4-e08c-40b9-96c5-d567bb78c089 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:45.702858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:45.722420 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1232/2440] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:45.728138 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68144c17-a5eb-4d60-bef8-65c87a24bf0b None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:45.728547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68144c17-a5eb-4d60-bef8-65c87a24bf0b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:45.728725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68144c17-a5eb-4d60-bef8-65c87a24bf0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:45.761482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-68144c17-a5eb-4d60-bef8-65c87a24bf0b None None] MFA Rules not processed for user `19d1751261ee4972add64c8e71a6bc47`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:45.767424 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-68144c17-a5eb-4d60-bef8-65c87a24bf0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:45.787222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a60b4-e08c-40b9-96c5-d567bb78c089 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:45.794084 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950878f-9f96-4bf0-af02-dc17a54fd9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:45.795530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950878f-9f96-4bf0-af02-dc17a54fd9e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:45.795881 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950878f-9f96-4bf0-af02-dc17a54fd9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:45.796634 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c950878f-9f96-4bf0-af02-dc17a54fd9e9 admin admin] PATH_INFO: `/v3/projects/91ee581f47a5419098d02102081f617d` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:45.797739 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950878f-9f96-4bf0-af02-dc17a54fd9e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=91ee581f47a5419098d02102081f617d)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:45.800445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c950878f-9f96-4bf0-af02-dc17a54fd9e9 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:45.830158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:45.840418 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1233/2441] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:45.840418 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea296020-0ab7-4d39-a437-09d5446fb104 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:45.840418 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea296020-0ab7-4d39-a437-09d5446fb104 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:45.840418 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea296020-0ab7-4d39-a437-09d5446fb104 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:45.862568 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-ea296020-0ab7-4d39-a437-09d5446fb104 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:45.863786 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ea296020-0ab7-4d39-a437-09d5446fb104 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:45.865558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c950878f-9f96-4bf0-af02-dc17a54fd9e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91ee581f47a5419098d02102081f617d', 'request_id': 'req-c950878f-9f96-4bf0-af02-dc17a54fd9e9'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.871878 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c950878f-9f96-4bf0-af02-dc17a54fd9e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91ee581f47a5419098d02102081f617d', 'request_id': 'req-c950878f-9f96-4bf0-af02-dc17a54fd9e9'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.878106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c950878f-9f96-4bf0-af02-dc17a54fd9e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91ee581f47a5419098d02102081f617d', 'request_id': 'req-c950878f-9f96-4bf0-af02-dc17a54fd9e9'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.892583 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c950878f-9f96-4bf0-af02-dc17a54fd9e9 admin admin] The token cache is being invalidate because project 91ee581f47a5419098d02102081f617d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:07:45.892883 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c950878f-9f96-4bf0-af02-dc17a54fd9e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.893537 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c950878f-9f96-4bf0-af02-dc17a54fd9e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:45.895158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1209/2442] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:07:45 2022] DELETE /identity/v3/projects/91ee581f47a5419098d02102081f617d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:07:45.895917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950878f-9f96-4bf0-af02-dc17a54fd9e9 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:45.897209 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c950878f-9f96-4bf0-af02-dc17a54fd9e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:45.927928 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1234/2443] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:45.929280 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319fa21-df41-4365-b520-2672062123d9 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:45.929642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319fa21-df41-4365-b520-2672062123d9 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:45.929995 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0319fa21-df41-4365-b520-2672062123d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:45.947633 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-0319fa21-df41-4365-b520-2672062123d9 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:45.949158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-0319fa21-df41-4365-b520-2672062123d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:45.957568 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1235/2444] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:45.958343 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319fa21-df41-4365-b520-2672062123d9 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:45.968032 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0319fa21-df41-4365-b520-2672062123d9 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:45.972819 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e4708f-4e7d-4af4-9c6e-ab76c37bf784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:45.973479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4708f-4e7d-4af4-9c6e-ab76c37bf784 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:45.973795 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4708f-4e7d-4af4-9c6e-ab76c37bf784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:45.974081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e4708f-4e7d-4af4-9c6e-ab76c37bf784 admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:45.975801 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e4708f-4e7d-4af4-9c6e-ab76c37bf784 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:07:45.976123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e4708f-4e7d-4af4-9c6e-ab76c37bf784 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:45.977753 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e4708f-4e7d-4af4-9c6e-ab76c37bf784 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:45.980135 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c950878f-9f96-4bf0-af02-dc17a54fd9e9 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:45.985959 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e992692-8361-4aed-9fea-5cb5e04b815b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:45.986764 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e992692-8361-4aed-9fea-5cb5e04b815b service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:45.987157 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e992692-8361-4aed-9fea-5cb5e04b815b service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:45.987563 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e992692-8361-4aed-9fea-5cb5e04b815b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:45.989478 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-9e992692-8361-4aed-9fea-5cb5e04b815b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:45.993353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1236/2445] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:07:45.994952 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736279f-0cb9-4570-8580-faeef2310e0d None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:45.995251 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736279f-0cb9-4570-8580-faeef2310e0d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:45.995536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a736279f-0cb9-4570-8580-faeef2310e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:46.016906 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a736279f-0cb9-4570-8580-faeef2310e0d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:46.018110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a736279f-0cb9-4570-8580-faeef2310e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:46.026844 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:46.027961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1237/2446] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:46.028874 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736279f-0cb9-4570-8580-faeef2310e0d None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:46.041295 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a736279f-0cb9-4570-8580-faeef2310e0d None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:46.046528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b33b2f-5ca5-48f3-86b7-773696310d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:46.047212 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b33b2f-5ca5-48f3-86b7-773696310d81 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:46.047496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b33b2f-5ca5-48f3-86b7-773696310d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:46.047776 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b33b2f-5ca5-48f3-86b7-773696310d81 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:46.048461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b33b2f-5ca5-48f3-86b7-773696310d81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:46.050386 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b33b2f-5ca5-48f3-86b7-773696310d81 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:46.059309 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e992692-8361-4aed-9fea-5cb5e04b815b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:46.060887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e992692-8361-4aed-9fea-5cb5e04b815b service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:46.071113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1210/2447] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:07:46.071915 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e992692-8361-4aed-9fea-5cb5e04b815b service nova] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:46.080472 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1238/2448] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:07:46 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:07:46.081473 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b33b2f-5ca5-48f3-86b7-773696310d81 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:46.084232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e992692-8361-4aed-9fea-5cb5e04b815b service nova] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:46.149346 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1362f4e-cce4-4cd5-be2e-71646ff6ecc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:46.150091 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1362f4e-cce4-4cd5-be2e-71646ff6ecc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:46.150445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1362f4e-cce4-4cd5-be2e-71646ff6ecc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:46.151350 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1362f4e-cce4-4cd5-be2e-71646ff6ecc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:46.152145 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1362f4e-cce4-4cd5-be2e-71646ff6ecc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:07:46.152526 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1362f4e-cce4-4cd5-be2e-71646ff6ecc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:46.154134 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1362f4e-cce4-4cd5-be2e-71646ff6ecc9 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:46.158296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b33b2f-5ca5-48f3-86b7-773696310d81 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:46.164922 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d01a7-e2dc-4f23-a793-1de66111ea30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:46.165653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d01a7-e2dc-4f23-a793-1de66111ea30 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:46.166171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d01a7-e2dc-4f23-a793-1de66111ea30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:46.166313 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1211/2449] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:07:46.166464 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407d01a7-e2dc-4f23-a793-1de66111ea30 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:46.167799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d01a7-e2dc-4f23-a793-1de66111ea30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:46.170071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407d01a7-e2dc-4f23-a793-1de66111ea30 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:46.170830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1362f4e-cce4-4cd5-be2e-71646ff6ecc9 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:46.183579 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1362f4e-cce4-4cd5-be2e-71646ff6ecc9 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:46.188534 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6726ec-14aa-4bdf-ab96-0f171c23e6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:46.189236 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6726ec-14aa-4bdf-ab96-0f171c23e6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:46.189583 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6726ec-14aa-4bdf-ab96-0f171c23e6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:46.189805 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6726ec-14aa-4bdf-ab96-0f171c23e6a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:46.190567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6726ec-14aa-4bdf-ab96-0f171c23e6a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:46.192022 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1239/2450] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:07:46 2022] POST /identity/v3/users => generated 286 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:07:46.192479 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6726ec-14aa-4bdf-ab96-0f171c23e6a3 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:46.196774 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d01a7-e2dc-4f23-a793-1de66111ea30 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:46.212955 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407d01a7-e2dc-4f23-a793-1de66111ea30 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:46.219545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355cddf-fe9c-4687-91aa-15ac8bb917da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:46.220322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355cddf-fe9c-4687-91aa-15ac8bb917da admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:46.220677 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355cddf-fe9c-4687-91aa-15ac8bb917da admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:46.221026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355cddf-fe9c-4687-91aa-15ac8bb917da admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:46.221925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1212/2451] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:07:46 2022] POST /identity/v3/projects => generated 365 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:07:46.225110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355cddf-fe9c-4687-91aa-15ac8bb917da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:46.225831 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6726ec-14aa-4bdf-ab96-0f171c23e6a3 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:46.227068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355cddf-fe9c-4687-91aa-15ac8bb917da admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:46.242947 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1240/2452] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:07:46.247242 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355cddf-fe9c-4687-91aa-15ac8bb917da admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:46.259056 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355cddf-fe9c-4687-91aa-15ac8bb917da admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:46.296352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:46.303466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6726ec-14aa-4bdf-ab96-0f171c23e6a3 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:46.308785 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25d764-604d-4dc0-a79b-5883a93e7283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:46.309463 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25d764-604d-4dc0-a79b-5883a93e7283 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:46.309751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25d764-604d-4dc0-a79b-5883a93e7283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:46.310031 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef25d764-604d-4dc0-a79b-5883a93e7283 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:46.310755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25d764-604d-4dc0-a79b-5883a93e7283 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:46.312871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef25d764-604d-4dc0-a79b-5883a93e7283 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:46.337378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1213/2453] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:07:46 2022] POST /identity/v3/users => generated 283 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:07:46.341938 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25d764-604d-4dc0-a79b-5883a93e7283 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:46.343708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6f636-1e20-4f6c-8c02-0a5cac5debbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:46.344897 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6f636-1e20-4f6c-8c02-0a5cac5debbd admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:46.345353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6f636-1e20-4f6c-8c02-0a5cac5debbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:46.345959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb6f636-1e20-4f6c-8c02-0a5cac5debbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:46.347124 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6f636-1e20-4f6c-8c02-0a5cac5debbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:46.349688 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb6f636-1e20-4f6c-8c02-0a5cac5debbd admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:46.353803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef25d764-604d-4dc0-a79b-5883a93e7283 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:46.359854 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b1ffc-34d6-4644-bbf1-fb149651d4f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:46.360670 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b1ffc-34d6-4644-bbf1-fb149651d4f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:46.361054 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b1ffc-34d6-4644-bbf1-fb149651d4f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:46.361433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6b1ffc-34d6-4644-bbf1-fb149651d4f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:46.361693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1241/2454] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:07:46.362492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b1ffc-34d6-4644-bbf1-fb149651d4f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:46.364680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6b1ffc-34d6-4644-bbf1-fb149651d4f0 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:46.367505 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6f636-1e20-4f6c-8c02-0a5cac5debbd admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:46.378844 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1214/2455] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:07:46.379877 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb6f636-1e20-4f6c-8c02-0a5cac5debbd admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:46.380072 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b1ffc-34d6-4644-bbf1-fb149651d4f0 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:46.389416 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f0c53b-84fb-4324-bb78-fb5b3659a2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:46.389998 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0c53b-84fb-4324-bb78-fb5b3659a2f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:46.390279 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0c53b-84fb-4324-bb78-fb5b3659a2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:46.391497 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f0c53b-84fb-4324-bb78-fb5b3659a2f5 admin admin] PATH_INFO: `/v3/projects/db3d6f4790e3407693f21f92a47f86c0/users/ea9d7afbe2cf437fbda5d6cb2464e4dc/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:46.391638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6b1ffc-34d6-4644-bbf1-fb149651d4f0 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:46.397182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c421d-2951-4b83-b610-c84195f25f80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:46.398005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c421d-2951-4b83-b610-c84195f25f80 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:46.398305 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c421d-2951-4b83-b610-c84195f25f80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:46.398608 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719c421d-2951-4b83-b610-c84195f25f80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:46.399482 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c421d-2951-4b83-b610-c84195f25f80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:46.401698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719c421d-2951-4b83-b610-c84195f25f80 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:46.412334 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0c53b-84fb-4324-bb78-fb5b3659a2f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db3d6f4790e3407693f21f92a47f86c0, user_id=ea9d7afbe2cf437fbda5d6cb2464e4dc, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:46.413497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1215/2456] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:07:46.417379 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c421d-2951-4b83-b610-c84195f25f80 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:46.418405 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f0c53b-84fb-4324-bb78-fb5b3659a2f5 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:46.434844 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719c421d-2951-4b83-b610-c84195f25f80 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:46.440265 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:46.449579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1242/2457] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:07:46 2022] PUT /identity/v3/projects/db3d6f4790e3407693f21f92a47f86c0/users/ea9d7afbe2cf437fbda5d6cb2464e4dc/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:07:46.459868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea7c1c-9358-4615-92d4-c42fc46576e7 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:46.460751 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea7c1c-9358-4615-92d4-c42fc46576e7 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:46.461090 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea7c1c-9358-4615-92d4-c42fc46576e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:46.493313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-91ea7c1c-9358-4615-92d4-c42fc46576e7 None None] MFA Rules not processed for user `ea9d7afbe2cf437fbda5d6cb2464e4dc`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:46.494571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-91ea7c1c-9358-4615-92d4-c42fc46576e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:46.516666 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b55cb3-efc1-4c9e-9600-89df8dbe6d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:46.517684 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b55cb3-efc1-4c9e-9600-89df8dbe6d36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:46.518023 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b55cb3-efc1-4c9e-9600-89df8dbe6d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:46.518353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b55cb3-efc1-4c9e-9600-89df8dbe6d36 admin admin] PATH_INFO: `/v3/projects/5e1ec76367d34267a52ac7921ee12730/users/bb33f5a8bce2481c896c9307338ca634/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:46.535323 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b55cb3-efc1-4c9e-9600-89df8dbe6d36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e1ec76367d34267a52ac7921ee12730, user_id=bb33f5a8bce2481c896c9307338ca634, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:46.538217 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b55cb3-efc1-4c9e-9600-89df8dbe6d36 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:46.557635 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1216/2458] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:07:46 2022] PUT /identity/v3/projects/5e1ec76367d34267a52ac7921ee12730/users/bb33f5a8bce2481c896c9307338ca634/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:46.568735 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af6cbf-8b16-49a6-a615-747ee534d3a9 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:46.569035 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af6cbf-8b16-49a6-a615-747ee534d3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:46.569321 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16af6cbf-8b16-49a6-a615-747ee534d3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:46.573615 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1243/2459] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:46.584399 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea7c1c-9358-4615-92d4-c42fc46576e7 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:46.585765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-91ea7c1c-9358-4615-92d4-c42fc46576e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:46.601928 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-16af6cbf-8b16-49a6-a615-747ee534d3a9 None None] MFA Rules not processed for user `bb33f5a8bce2481c896c9307338ca634`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:46.603607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-16af6cbf-8b16-49a6-a615-747ee534d3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:46.669392 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea7c1c-9358-4615-92d4-c42fc46576e7 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:46.681414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1217/2460] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:46 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:07:46.695911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af6cbf-8b16-49a6-a615-747ee534d3a9 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:46.698417 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-16af6cbf-8b16-49a6-a615-747ee534d3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:46.702618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:46.747407 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831f8e76-62b0-418a-b0f2-dc1688e9b104 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:46.748161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f8e76-62b0-418a-b0f2-dc1688e9b104 service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:46.748446 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f8e76-62b0-418a-b0f2-dc1688e9b104 service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:46.748756 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831f8e76-62b0-418a-b0f2-dc1688e9b104 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:46.763597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16af6cbf-8b16-49a6-a615-747ee534d3a9 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:46.767203 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31782d-0d90-4402-bb8d-82b4ca531bc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:46.767843 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31782d-0d90-4402-bb8d-82b4ca531bc5 service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:46.768127 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31782d-0d90-4402-bb8d-82b4ca531bc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:46.768406 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb31782d-0d90-4402-bb8d-82b4ca531bc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:46.776649 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31782d-0d90-4402-bb8d-82b4ca531bc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:46.778067 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb31782d-0d90-4402-bb8d-82b4ca531bc5 service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:46.786952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1218/2461] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:46 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:07:46.804843 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f8e76-62b0-418a-b0f2-dc1688e9b104 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:46.806488 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831f8e76-62b0-418a-b0f2-dc1688e9b104 service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:46.815979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1244/2462] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:07:51.733023 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31782d-0d90-4402-bb8d-82b4ca531bc5 service cinder] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:51.761640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:51.809459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb31782d-0d90-4402-bb8d-82b4ca531bc5 service cinder] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:51.814285 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b50f60-1e19-417f-98ce-0e2b39be9aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:51.815206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b50f60-1e19-417f-98ce-0e2b39be9aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:51.815495 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b50f60-1e19-417f-98ce-0e2b39be9aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:51.815808 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b50f60-1e19-417f-98ce-0e2b39be9aa3 admin admin] PATH_INFO: `/v3/users/ea9d7afbe2cf437fbda5d6cb2464e4dc` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:51.817003 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b50f60-1e19-417f-98ce-0e2b39be9aa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea9d7afbe2cf437fbda5d6cb2464e4dc)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:51.818963 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b50f60-1e19-417f-98ce-0e2b39be9aa3 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:51.876594 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-25b50f60-1e19-417f-98ce-0e2b39be9aa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea9d7afbe2cf437fbda5d6cb2464e4dc'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:51.877669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-25b50f60-1e19-417f-98ce-0e2b39be9aa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea9d7afbe2cf437fbda5d6cb2464e4dc'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:51.895127 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-25b50f60-1e19-417f-98ce-0e2b39be9aa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea9d7afbe2cf437fbda5d6cb2464e4dc'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:51.896198 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-25b50f60-1e19-417f-98ce-0e2b39be9aa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea9d7afbe2cf437fbda5d6cb2464e4dc'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:51.910882 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-25b50f60-1e19-417f-98ce-0e2b39be9aa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea9d7afbe2cf437fbda5d6cb2464e4dc'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:51.927235 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1219/2463] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:07:51 2022] DELETE /identity/v3/users/ea9d7afbe2cf437fbda5d6cb2464e4dc => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:51.932408 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1219ecc-c5e0-4ffa-a069-cb5b845c24aa None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:51.932895 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1219ecc-c5e0-4ffa-a069-cb5b845c24aa None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:51.932993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1219ecc-c5e0-4ffa-a069-cb5b845c24aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:51.952738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-b1219ecc-c5e0-4ffa-a069-cb5b845c24aa None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:51.953957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b1219ecc-c5e0-4ffa-a069-cb5b845c24aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:52.017530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1245/2464] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:52.027798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b50f60-1e19-417f-98ce-0e2b39be9aa3 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:52.029189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-25b50f60-1e19-417f-98ce-0e2b39be9aa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:52.096068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b50f60-1e19-417f-98ce-0e2b39be9aa3 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:52.100264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63598a04-14ff-40ca-ba0b-ac63ffe67fac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:52.100950 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63598a04-14ff-40ca-ba0b-ac63ffe67fac service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:52.101235 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63598a04-14ff-40ca-ba0b-ac63ffe67fac service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:52.101512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63598a04-14ff-40ca-ba0b-ac63ffe67fac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:52.111723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63598a04-14ff-40ca-ba0b-ac63ffe67fac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:52.113207 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63598a04-14ff-40ca-ba0b-ac63ffe67fac service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:52.123961 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1220/2465] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:07:52.408297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1219ecc-c5e0-4ffa-a069-cb5b845c24aa None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:52.409912 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b1219ecc-c5e0-4ffa-a069-cb5b845c24aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:52.425001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1219ecc-c5e0-4ffa-a069-cb5b845c24aa None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:52.431261 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf3ab8a-fea1-4245-9d36-5414f146f869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:52.432133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3ab8a-fea1-4245-9d36-5414f146f869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:52.432410 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3ab8a-fea1-4245-9d36-5414f146f869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:52.432719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf3ab8a-fea1-4245-9d36-5414f146f869 admin admin] PATH_INFO: `/v3/projects/db3d6f4790e3407693f21f92a47f86c0` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:52.433968 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3ab8a-fea1-4245-9d36-5414f146f869 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db3d6f4790e3407693f21f92a47f86c0)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:52.435897 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf3ab8a-fea1-4245-9d36-5414f146f869 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:52.486104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-daf3ab8a-fea1-4245-9d36-5414f146f869 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db3d6f4790e3407693f21f92a47f86c0', 'request_id': 'req-daf3ab8a-fea1-4245-9d36-5414f146f869'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:52.492519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-daf3ab8a-fea1-4245-9d36-5414f146f869 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db3d6f4790e3407693f21f92a47f86c0', 'request_id': 'req-daf3ab8a-fea1-4245-9d36-5414f146f869'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:52.498218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-daf3ab8a-fea1-4245-9d36-5414f146f869 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db3d6f4790e3407693f21f92a47f86c0', 'request_id': 'req-daf3ab8a-fea1-4245-9d36-5414f146f869'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:52.499933 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:52.512097 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-daf3ab8a-fea1-4245-9d36-5414f146f869 admin admin] The token cache is being invalidate because project db3d6f4790e3407693f21f92a47f86c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:07:52.512386 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-daf3ab8a-fea1-4245-9d36-5414f146f869 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:52.513015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-daf3ab8a-fea1-4245-9d36-5414f146f869 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:52.514848 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1246/2466] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:07:52 2022] DELETE /identity/v3/projects/db3d6f4790e3407693f21f92a47f86c0 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:07:52.520052 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da7b92-ae59-4e22-b023-ad95517cee92 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:52.520539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da7b92-ae59-4e22-b023-ad95517cee92 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:52.521152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da7b92-ae59-4e22-b023-ad95517cee92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:52.539489 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-a9da7b92-ae59-4e22-b023-ad95517cee92 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:52.541007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-a9da7b92-ae59-4e22-b023-ad95517cee92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:52.583978 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:52.603750 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1221/2467] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:52.611491 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b7fb4-15ea-4a00-baba-1180e058bc80 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:52.611786 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b7fb4-15ea-4a00-baba-1180e058bc80 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:52.612067 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7b7fb4-15ea-4a00-baba-1180e058bc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:52.631216 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-fe7b7fb4-15ea-4a00-baba-1180e058bc80 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:52.632405 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-fe7b7fb4-15ea-4a00-baba-1180e058bc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:52.640731 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1247/2468] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:52.644764 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da7b92-ae59-4e22-b023-ad95517cee92 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:52.654427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da7b92-ae59-4e22-b023-ad95517cee92 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:52.659017 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f3b8d-6c54-46cb-bd0d-7f1e3f64dcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:52.659664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f3b8d-6c54-46cb-bd0d-7f1e3f64dcf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:52.659950 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f3b8d-6c54-46cb-bd0d-7f1e3f64dcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:52.660227 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179f3b8d-6c54-46cb-bd0d-7f1e3f64dcf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:52.660944 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f3b8d-6c54-46cb-bd0d-7f1e3f64dcf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:07:52.661254 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f3b8d-6c54-46cb-bd0d-7f1e3f64dcf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:52.662866 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179f3b8d-6c54-46cb-bd0d-7f1e3f64dcf2 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:52.674351 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1222/2469] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:07:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:07:52.678501 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b7fb4-15ea-4a00-baba-1180e058bc80 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:52.688120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7b7fb4-15ea-4a00-baba-1180e058bc80 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:52.692763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee84ed-6d13-45eb-a321-8d50716b87fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:52.693403 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee84ed-6d13-45eb-a321-8d50716b87fe admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:52.693686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee84ed-6d13-45eb-a321-8d50716b87fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:52.693962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ee84ed-6d13-45eb-a321-8d50716b87fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:52.694701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee84ed-6d13-45eb-a321-8d50716b87fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:52.696577 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ee84ed-6d13-45eb-a321-8d50716b87fe admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:52.723530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1248/2470] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:07:52 2022] POST /identity/v3/projects => generated 363 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:07:52.727686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f3b8d-6c54-46cb-bd0d-7f1e3f64dcf2 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:52.737410 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179f3b8d-6c54-46cb-bd0d-7f1e3f64dcf2 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:52.792030 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317ef5e-22de-4ae5-8f06-e1d2fbe5ecfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:52.792696 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317ef5e-22de-4ae5-8f06-e1d2fbe5ecfa admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:52.792980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317ef5e-22de-4ae5-8f06-e1d2fbe5ecfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:52.793255 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0317ef5e-22de-4ae5-8f06-e1d2fbe5ecfa admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:52.793920 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317ef5e-22de-4ae5-8f06-e1d2fbe5ecfa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:52.795847 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0317ef5e-22de-4ae5-8f06-e1d2fbe5ecfa admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:52.814930 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1223/2471] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:07:52 2022] POST /identity/v3/users => generated 282 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:07:52.819195 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee84ed-6d13-45eb-a321-8d50716b87fe admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:52.829663 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ee84ed-6d13-45eb-a321-8d50716b87fe admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:52.834295 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323aafb7-cbac-4047-9db6-d8948e02ed58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:52.834980 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323aafb7-cbac-4047-9db6-d8948e02ed58 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:52.835272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323aafb7-cbac-4047-9db6-d8948e02ed58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:52.835552 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323aafb7-cbac-4047-9db6-d8948e02ed58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:52.836266 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323aafb7-cbac-4047-9db6-d8948e02ed58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:52.838352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323aafb7-cbac-4047-9db6-d8948e02ed58 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:52.848557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1249/2472] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:07:52.852297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317ef5e-22de-4ae5-8f06-e1d2fbe5ecfa admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:52.862760 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0317ef5e-22de-4ae5-8f06-e1d2fbe5ecfa admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:52.867341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c5f4a-7ab1-4c20-9f6e-74283f270a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:52.867990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c5f4a-7ab1-4c20-9f6e-74283f270a1b admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:52.868277 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c5f4a-7ab1-4c20-9f6e-74283f270a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:52.868557 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c5f4a-7ab1-4c20-9f6e-74283f270a1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:52.869779 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c5f4a-7ab1-4c20-9f6e-74283f270a1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:52.871397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c5f4a-7ab1-4c20-9f6e-74283f270a1b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:52.881239 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1224/2473] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:07:52.885679 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323aafb7-cbac-4047-9db6-d8948e02ed58 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:52.895477 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323aafb7-cbac-4047-9db6-d8948e02ed58 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:52.900054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd49d68-4151-4382-8752-669cb7311004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:52.900958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd49d68-4151-4382-8752-669cb7311004 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:52.901246 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd49d68-4151-4382-8752-669cb7311004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:52.901523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd49d68-4151-4382-8752-669cb7311004 admin admin] PATH_INFO: `/v3/projects/ea8fed976c5a45168625ccbbf99ab22b/users/8ac13dbf9cdb4401a935a7590d989739/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:52.915737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd49d68-4151-4382-8752-669cb7311004 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea8fed976c5a45168625ccbbf99ab22b, user_id=8ac13dbf9cdb4401a935a7590d989739, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:52.918377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd49d68-4151-4382-8752-669cb7311004 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:52.932054 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1250/2474] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:07:52 2022] PUT /identity/v3/projects/ea8fed976c5a45168625ccbbf99ab22b/users/8ac13dbf9cdb4401a935a7590d989739/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:52.943325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667dcc4-e1fd-4406-aa09-f2ef6835df88 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:52.943623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667dcc4-e1fd-4406-aa09-f2ef6835df88 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:52.943907 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7667dcc4-e1fd-4406-aa09-f2ef6835df88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:52.971011 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-7667dcc4-e1fd-4406-aa09-f2ef6835df88 None None] MFA Rules not processed for user `8ac13dbf9cdb4401a935a7590d989739`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:52.972219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-7667dcc4-e1fd-4406-aa09-f2ef6835df88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:53.037833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1225/2475] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:52 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:53.042957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd49d68-4151-4382-8752-669cb7311004 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:53.131040 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd49d68-4151-4382-8752-669cb7311004 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:53.137942 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d97623-2fc3-49eb-adab-d397456e2515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:53.138783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d97623-2fc3-49eb-adab-d397456e2515 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:53.139226 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d97623-2fc3-49eb-adab-d397456e2515 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:53.139516 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d97623-2fc3-49eb-adab-d397456e2515 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:53.140218 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d97623-2fc3-49eb-adab-d397456e2515 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:53.142224 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d97623-2fc3-49eb-adab-d397456e2515 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:53.168065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1251/2476] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:07:53 2022] POST /identity/v3/projects => generated 365 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:07:53.171990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7667dcc4-e1fd-4406-aa09-f2ef6835df88 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:53.183855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7667dcc4-e1fd-4406-aa09-f2ef6835df88 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:53.224647 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:53.243583 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afcae3-f8e2-492a-a5c2-2fc628d532cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:53.244816 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afcae3-f8e2-492a-a5c2-2fc628d532cf admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:53.245104 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afcae3-f8e2-492a-a5c2-2fc628d532cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:53.245384 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4afcae3-f8e2-492a-a5c2-2fc628d532cf admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:53.246078 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afcae3-f8e2-492a-a5c2-2fc628d532cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:53.248039 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4afcae3-f8e2-492a-a5c2-2fc628d532cf admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:53.267798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1226/2477] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:07:53 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:07:53.272269 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d97623-2fc3-49eb-adab-d397456e2515 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:53.283358 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d97623-2fc3-49eb-adab-d397456e2515 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:53.288124 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ab113-0af7-4494-87b5-b05fffb4039e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:53.288789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ab113-0af7-4494-87b5-b05fffb4039e admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:53.289078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ab113-0af7-4494-87b5-b05fffb4039e admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:53.289358 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7ab113-0af7-4494-87b5-b05fffb4039e admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:53.290145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ab113-0af7-4494-87b5-b05fffb4039e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:53.291873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7ab113-0af7-4494-87b5-b05fffb4039e admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:53.302854 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1252/2478] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:07:53.306934 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afcae3-f8e2-492a-a5c2-2fc628d532cf admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:53.318245 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4afcae3-f8e2-492a-a5c2-2fc628d532cf admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:53.324135 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c49a5-8fcc-46bc-a923-049d6424d98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:53.325103 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c49a5-8fcc-46bc-a923-049d6424d98c admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:53.326459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c49a5-8fcc-46bc-a923-049d6424d98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:53.326459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c49a5-8fcc-46bc-a923-049d6424d98c admin admin] PATH_INFO: `/v3/projects/4375665e8d4644569083471a41b9dcc2/users/c05662cd54bc4a26a2db024fedfeed37/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:53.343727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c49a5-8fcc-46bc-a923-049d6424d98c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4375665e8d4644569083471a41b9dcc2, user_id=c05662cd54bc4a26a2db024fedfeed37, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:53.346992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c49a5-8fcc-46bc-a923-049d6424d98c admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:53.359880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1227/2479] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:07:53 2022] PUT /identity/v3/projects/4375665e8d4644569083471a41b9dcc2/users/c05662cd54bc4a26a2db024fedfeed37/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:07:53.364249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ab113-0af7-4494-87b5-b05fffb4039e admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:53.375313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7ab113-0af7-4494-87b5-b05fffb4039e admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:53.435466 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9905a699-8ece-4188-8c1d-e3e621eb6660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:53.436223 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905a699-8ece-4188-8c1d-e3e621eb6660 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:53.436515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905a699-8ece-4188-8c1d-e3e621eb6660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:53.436796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9905a699-8ece-4188-8c1d-e3e621eb6660 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:53.437567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905a699-8ece-4188-8c1d-e3e621eb6660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:53.439451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9905a699-8ece-4188-8c1d-e3e621eb6660 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:53.441832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:53.453767 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1253/2480] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:07:53.457761 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c49a5-8fcc-46bc-a923-049d6424d98c admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:53.468302 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c49a5-8fcc-46bc-a923-049d6424d98c admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:53.473305 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12bba6-15eb-4342-8c93-b5db36d08839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:53.474240 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12bba6-15eb-4342-8c93-b5db36d08839 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:53.474505 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12bba6-15eb-4342-8c93-b5db36d08839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:53.474822 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb12bba6-15eb-4342-8c93-b5db36d08839 admin admin] PATH_INFO: `/v3/projects/4375665e8d4644569083471a41b9dcc2/users/c05662cd54bc4a26a2db024fedfeed37/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:53.478569 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb12bba6-15eb-4342-8c93-b5db36d08839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4375665e8d4644569083471a41b9dcc2, user_id=c05662cd54bc4a26a2db024fedfeed37, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:53.483210 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb12bba6-15eb-4342-8c93-b5db36d08839 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:53.494136 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1228/2481] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:07:53 2022] PUT /identity/v3/projects/4375665e8d4644569083471a41b9dcc2/users/c05662cd54bc4a26a2db024fedfeed37/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:53.504121 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84bdee-2433-4b1f-8679-1c6f026fe746 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:53.504418 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84bdee-2433-4b1f-8679-1c6f026fe746 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:53.504704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f84bdee-2433-4b1f-8679-1c6f026fe746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:53.538165 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-7f84bdee-2433-4b1f-8679-1c6f026fe746 None None] MFA Rules not processed for user `c05662cd54bc4a26a2db024fedfeed37`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:53.539407 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7f84bdee-2433-4b1f-8679-1c6f026fe746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:53.574535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12bba6-15eb-4342-8c93-b5db36d08839 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:53.575385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-fb12bba6-15eb-4342-8c93-b5db36d08839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:53.616113 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1254/2482] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:53 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:07:53.628133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84bdee-2433-4b1f-8679-1c6f026fe746 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:53.629418 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7f84bdee-2433-4b1f-8679-1c6f026fe746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:53.640713 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb12bba6-15eb-4342-8c93-b5db36d08839 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:53.645636 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b739a6a-e821-47a5-a99f-9c198457e99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:53.646659 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b739a6a-e821-47a5-a99f-9c198457e99b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:53.646972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b739a6a-e821-47a5-a99f-9c198457e99b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:53.647255 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b739a6a-e821-47a5-a99f-9c198457e99b admin admin] PATH_INFO: `/v3/users/19d1751261ee4972add64c8e71a6bc47` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:53.648566 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b739a6a-e821-47a5-a99f-9c198457e99b admin admin] RBAC: Authorizing `identity:delete_user(user_id=19d1751261ee4972add64c8e71a6bc47)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:53.650552 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b739a6a-e821-47a5-a99f-9c198457e99b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:53.687618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f84bdee-2433-4b1f-8679-1c6f026fe746 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:53.691315 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56ecd64-6352-48da-a5de-8af582ddb3b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:53.691972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ecd64-6352-48da-a5de-8af582ddb3b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:53.692268 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ecd64-6352-48da-a5de-8af582ddb3b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:53.692546 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56ecd64-6352-48da-a5de-8af582ddb3b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:53.697552 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5b739a6a-e821-47a5-a99f-9c198457e99b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19d1751261ee4972add64c8e71a6bc47'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:53.698461 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5b739a6a-e821-47a5-a99f-9c198457e99b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19d1751261ee4972add64c8e71a6bc47'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:53.702327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ecd64-6352-48da-a5de-8af582ddb3b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:53.703773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56ecd64-6352-48da-a5de-8af582ddb3b6 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:53.705426 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5b739a6a-e821-47a5-a99f-9c198457e99b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19d1751261ee4972add64c8e71a6bc47'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:53.706071 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5b739a6a-e821-47a5-a99f-9c198457e99b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19d1751261ee4972add64c8e71a6bc47'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:53.713178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1255/2483] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:53 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:07:53.717957 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5b739a6a-e821-47a5-a99f-9c198457e99b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19d1751261ee4972add64c8e71a6bc47'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:53.730398 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1229/2484] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:07:53 2022] DELETE /identity/v3/users/19d1751261ee4972add64c8e71a6bc47 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:53.734737 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4a3e1-4770-47b2-8b3b-dbb3436ed3c3 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:53.735163 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4a3e1-4770-47b2-8b3b-dbb3436ed3c3 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:53.735611 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4a3e1-4770-47b2-8b3b-dbb3436ed3c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:53.754612 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-feb4a3e1-4770-47b2-8b3b-dbb3436ed3c3 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:53.755929 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-feb4a3e1-4770-47b2-8b3b-dbb3436ed3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:53.823567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1256/2485] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:53.836720 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b739a6a-e821-47a5-a99f-9c198457e99b admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:53.838230 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-5b739a6a-e821-47a5-a99f-9c198457e99b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:53.852834 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:53.902920 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b739a6a-e821-47a5-a99f-9c198457e99b admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:53.906680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fd4e1-245c-4438-8d79-4ea1cce73591 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:53.907361 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fd4e1-245c-4438-8d79-4ea1cce73591 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:53.907672 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fd4e1-245c-4438-8d79-4ea1cce73591 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:53.907941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fd4e1-245c-4438-8d79-4ea1cce73591 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:53.918386 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7fd4e1-245c-4438-8d79-4ea1cce73591 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:53.919935 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7fd4e1-245c-4438-8d79-4ea1cce73591 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:53.930019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1230/2486] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:07:54.130023 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4a3e1-4770-47b2-8b3b-dbb3436ed3c3 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:54.131575 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-feb4a3e1-4770-47b2-8b3b-dbb3436ed3c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:54.144606 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4a3e1-4770-47b2-8b3b-dbb3436ed3c3 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:54.151451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdac97-992f-43cd-9c8c-4c500312a6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:54.152794 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdac97-992f-43cd-9c8c-4c500312a6ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:54.153184 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdac97-992f-43cd-9c8c-4c500312a6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:54.153571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdac97-992f-43cd-9c8c-4c500312a6ea admin admin] PATH_INFO: `/v3/users/8ac13dbf9cdb4401a935a7590d989739` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:54.154994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdac97-992f-43cd-9c8c-4c500312a6ea admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ac13dbf9cdb4401a935a7590d989739)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:54.156988 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdac97-992f-43cd-9c8c-4c500312a6ea admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:54.188863 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:54.226796 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fd4e1-245c-4438-8d79-4ea1cce73591 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:54.229177 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-3f7fd4e1-245c-4438-8d79-4ea1cce73591 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:54.239088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-cebdac97-992f-43cd-9c8c-4c500312a6ea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ac13dbf9cdb4401a935a7590d989739'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:54.249743 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-cebdac97-992f-43cd-9c8c-4c500312a6ea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ac13dbf9cdb4401a935a7590d989739'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:54.251967 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-cebdac97-992f-43cd-9c8c-4c500312a6ea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ac13dbf9cdb4401a935a7590d989739'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:54.252315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fd4e1-245c-4438-8d79-4ea1cce73591 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:54.261207 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac26ba4-df35-4c69-b894-43c6052a87cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:54.262158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac26ba4-df35-4c69-b894-43c6052a87cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:54.262665 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac26ba4-df35-4c69-b894-43c6052a87cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:54.263015 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac26ba4-df35-4c69-b894-43c6052a87cf admin admin] PATH_INFO: `/v3/projects/eb2bd01b77f948ceb4ac08cc658d39fb` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:54.264502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac26ba4-df35-4c69-b894-43c6052a87cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb2bd01b77f948ceb4ac08cc658d39fb)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:54.266812 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac26ba4-df35-4c69-b894-43c6052a87cf admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:54.275470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-cebdac97-992f-43cd-9c8c-4c500312a6ea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ac13dbf9cdb4401a935a7590d989739'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:54.276586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-cebdac97-992f-43cd-9c8c-4c500312a6ea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ac13dbf9cdb4401a935a7590d989739'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:54.293297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1257/2487] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:07:54 2022] DELETE /identity/v3/users/8ac13dbf9cdb4401a935a7590d989739 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:54.294251 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdac97-992f-43cd-9c8c-4c500312a6ea admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:54.295873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-cebdac97-992f-43cd-9c8c-4c500312a6ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:54.325721 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-fac26ba4-df35-4c69-b894-43c6052a87cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb2bd01b77f948ceb4ac08cc658d39fb', 'request_id': 'req-fac26ba4-df35-4c69-b894-43c6052a87cf'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:54.332766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-fac26ba4-df35-4c69-b894-43c6052a87cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb2bd01b77f948ceb4ac08cc658d39fb', 'request_id': 'req-fac26ba4-df35-4c69-b894-43c6052a87cf'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:54.341043 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-fac26ba4-df35-4c69-b894-43c6052a87cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb2bd01b77f948ceb4ac08cc658d39fb', 'request_id': 'req-fac26ba4-df35-4c69-b894-43c6052a87cf'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:54.357535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-fac26ba4-df35-4c69-b894-43c6052a87cf admin admin] The token cache is being invalidate because project eb2bd01b77f948ceb4ac08cc658d39fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:07:54.357982 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-fac26ba4-df35-4c69-b894-43c6052a87cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:54.358718 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-fac26ba4-df35-4c69-b894-43c6052a87cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:54.360480 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1231/2488] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:07:54 2022] DELETE /identity/v3/projects/eb2bd01b77f948ceb4ac08cc658d39fb => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:07:54.364721 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73545523-528f-4e2f-b855-4ed9fe7426aa None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:54.365125 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73545523-528f-4e2f-b855-4ed9fe7426aa None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:54.365567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdac97-992f-43cd-9c8c-4c500312a6ea admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:54.365710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73545523-528f-4e2f-b855-4ed9fe7426aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:54.386005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-73545523-528f-4e2f-b855-4ed9fe7426aa None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:54.387340 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-73545523-528f-4e2f-b855-4ed9fe7426aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:54.430476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3709715-4b25-4248-9122-a5b43a5111fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:54.431625 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3709715-4b25-4248-9122-a5b43a5111fa service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:54.431905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3709715-4b25-4248-9122-a5b43a5111fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:54.432181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3709715-4b25-4248-9122-a5b43a5111fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:54.434147 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e3709715-4b25-4248-9122-a5b43a5111fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:54.462052 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1232/2489] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:54.464412 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0b96f-45ad-4a33-9884-976958237ae8 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:54.464723 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0b96f-45ad-4a33-9884-976958237ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:54.465006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0b96f-45ad-4a33-9884-976958237ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:54.498269 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-96a0b96f-45ad-4a33-9884-976958237ae8 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:54.499893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-96a0b96f-45ad-4a33-9884-976958237ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:54.509733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1233/2490] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:54.511046 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0b96f-45ad-4a33-9884-976958237ae8 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:54.514106 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3709715-4b25-4248-9122-a5b43a5111fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:54.515579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3709715-4b25-4248-9122-a5b43a5111fa service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:54.516964 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-96a0b96f-45ad-4a33-9884-976958237ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:54.527662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1258/2491] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:07:54.528968 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52d570-a073-4b1e-9cad-3a74acd178b7 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:54.529276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52d570-a073-4b1e-9cad-3a74acd178b7 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:54.529599 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b52d570-a073-4b1e-9cad-3a74acd178b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:54.539263 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0b96f-45ad-4a33-9884-976958237ae8 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:54.540420 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:54.545043 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561cca3-a9b9-49f9-9af7-b722fd1bbe90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:54.546213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561cca3-a9b9-49f9-9af7-b722fd1bbe90 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:54.546957 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561cca3-a9b9-49f9-9af7-b722fd1bbe90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:54.547341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f561cca3-a9b9-49f9-9af7-b722fd1bbe90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:54.554705 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-9b52d570-a073-4b1e-9cad-3a74acd178b7 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:54.557349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-9b52d570-a073-4b1e-9cad-3a74acd178b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:54.560805 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561cca3-a9b9-49f9-9af7-b722fd1bbe90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:54.562711 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f561cca3-a9b9-49f9-9af7-b722fd1bbe90 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:54.566877 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1259/2492] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:54.570841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52d570-a073-4b1e-9cad-3a74acd178b7 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:54.576640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1234/2493] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:07:54.582826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b52d570-a073-4b1e-9cad-3a74acd178b7 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:54.588363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73575f7f-fde2-4b77-83ed-39ad13be58d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:54.590634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73575f7f-fde2-4b77-83ed-39ad13be58d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:54.590634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73575f7f-fde2-4b77-83ed-39ad13be58d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:54.590634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73575f7f-fde2-4b77-83ed-39ad13be58d2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:54.590634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73575f7f-fde2-4b77-83ed-39ad13be58d2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:07:54.591254 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73575f7f-fde2-4b77-83ed-39ad13be58d2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:54.592934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73575f7f-fde2-4b77-83ed-39ad13be58d2 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:54.606844 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1260/2494] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:07:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 23:07:54.611102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561cca3-a9b9-49f9-9af7-b722fd1bbe90 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:54.623144 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f561cca3-a9b9-49f9-9af7-b722fd1bbe90 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:54.628550 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780c325-57ac-4f1b-8a0c-40dba8f4c154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:54.629223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780c325-57ac-4f1b-8a0c-40dba8f4c154 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:54.629514 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780c325-57ac-4f1b-8a0c-40dba8f4c154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:54.629815 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7780c325-57ac-4f1b-8a0c-40dba8f4c154 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:54.630489 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780c325-57ac-4f1b-8a0c-40dba8f4c154 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:54.632446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7780c325-57ac-4f1b-8a0c-40dba8f4c154 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:54.654643 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:54.666857 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1235/2495] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:07:54 2022] POST /identity/v3/projects => generated 375 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:07:54.671927 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73575f7f-fde2-4b77-83ed-39ad13be58d2 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:54.685471 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73575f7f-fde2-4b77-83ed-39ad13be58d2 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:54.756224 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3a769-4c48-4367-897a-386d38374e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:54.757189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3a769-4c48-4367-897a-386d38374e21 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:54.757723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3a769-4c48-4367-897a-386d38374e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:54.758160 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d3a769-4c48-4367-897a-386d38374e21 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:54.760110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3a769-4c48-4367-897a-386d38374e21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:54.763417 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d3a769-4c48-4367-897a-386d38374e21 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:54.787671 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1261/2496] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:07:54 2022] POST /identity/v3/users => generated 288 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:07:54.792742 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780c325-57ac-4f1b-8a0c-40dba8f4c154 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:54.803863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7780c325-57ac-4f1b-8a0c-40dba8f4c154 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:54.811045 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6b693-e443-48f1-b82c-b65eadf93491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:54.812191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6b693-e443-48f1-b82c-b65eadf93491 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:54.812558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6b693-e443-48f1-b82c-b65eadf93491 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:54.812857 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f6b693-e443-48f1-b82c-b65eadf93491 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:54.813993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f6b693-e443-48f1-b82c-b65eadf93491 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:54.816440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f6b693-e443-48f1-b82c-b65eadf93491 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:54.834886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1236/2497] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:07:54.836396 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3a769-4c48-4367-897a-386d38374e21 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:54.848782 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d3a769-4c48-4367-897a-386d38374e21 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:54.854716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899f1ec-bee1-4a5c-8337-0ba7da0e9c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:54.855463 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899f1ec-bee1-4a5c-8337-0ba7da0e9c21 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:54.855815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899f1ec-bee1-4a5c-8337-0ba7da0e9c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:54.856158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899f1ec-bee1-4a5c-8337-0ba7da0e9c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:54.857000 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1899f1ec-bee1-4a5c-8337-0ba7da0e9c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:54.858716 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1899f1ec-bee1-4a5c-8337-0ba7da0e9c21 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:54.873992 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1262/2498] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:07:54.875272 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6b693-e443-48f1-b82c-b65eadf93491 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:54.889937 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f6b693-e443-48f1-b82c-b65eadf93491 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:54.895278 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef15714-34de-4ae7-95f5-cf0e6c5c0967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:54.896219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef15714-34de-4ae7-95f5-cf0e6c5c0967 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:54.896559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef15714-34de-4ae7-95f5-cf0e6c5c0967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:54.896861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef15714-34de-4ae7-95f5-cf0e6c5c0967 admin admin] PATH_INFO: `/v3/projects/f84064db84a441a1a146641cf69efda1/users/230d5619188b4d27980be714b901c831/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:54.908779 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899f1ec-bee1-4a5c-8337-0ba7da0e9c21 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:54.910660 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-1899f1ec-bee1-4a5c-8337-0ba7da0e9c21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:54.916607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef15714-34de-4ae7-95f5-cf0e6c5c0967 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f84064db84a441a1a146641cf69efda1, user_id=230d5619188b4d27980be714b901c831, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:54.919811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef15714-34de-4ae7-95f5-cf0e6c5c0967 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:54.940377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899f1ec-bee1-4a5c-8337-0ba7da0e9c21 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:54.946683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1237/2499] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:07:54 2022] PUT /identity/v3/projects/f84064db84a441a1a146641cf69efda1/users/230d5619188b4d27980be714b901c831/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:54.954554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9b0b9-e023-4926-8362-b112720dead3 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:54.954902 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9b0b9-e023-4926-8362-b112720dead3 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:54.955200 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c9b0b9-e023-4926-8362-b112720dead3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:54.988423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-65c9b0b9-e023-4926-8362-b112720dead3 None None] MFA Rules not processed for user `230d5619188b4d27980be714b901c831`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:54.989802 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-65c9b0b9-e023-4926-8362-b112720dead3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:55.015889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641aa0a-f28d-451a-afac-ad80588e1c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:55.016763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641aa0a-f28d-451a-afac-ad80588e1c08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:55.017056 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641aa0a-f28d-451a-afac-ad80588e1c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:55.017336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641aa0a-f28d-451a-afac-ad80588e1c08 admin admin] PATH_INFO: `/v3/projects/ea8fed976c5a45168625ccbbf99ab22b` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:55.018514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641aa0a-f28d-451a-afac-ad80588e1c08 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea8fed976c5a45168625ccbbf99ab22b)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:55.020970 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641aa0a-f28d-451a-afac-ad80588e1c08 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:55.069453 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1238/2500] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:07:55.078527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5641aa0a-f28d-451a-afac-ad80588e1c08 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea8fed976c5a45168625ccbbf99ab22b', 'request_id': 'req-5641aa0a-f28d-451a-afac-ad80588e1c08'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.083700 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9b0b9-e023-4926-8362-b112720dead3 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:55.085812 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-65c9b0b9-e023-4926-8362-b112720dead3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:55.092027 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5641aa0a-f28d-451a-afac-ad80588e1c08 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea8fed976c5a45168625ccbbf99ab22b', 'request_id': 'req-5641aa0a-f28d-451a-afac-ad80588e1c08'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.099270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5641aa0a-f28d-451a-afac-ad80588e1c08 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea8fed976c5a45168625ccbbf99ab22b', 'request_id': 'req-5641aa0a-f28d-451a-afac-ad80588e1c08'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.102408 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:55.116249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5641aa0a-f28d-451a-afac-ad80588e1c08 admin admin] The token cache is being invalidate because project ea8fed976c5a45168625ccbbf99ab22b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:07:55.116741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5641aa0a-f28d-451a-afac-ad80588e1c08 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.120051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5641aa0a-f28d-451a-afac-ad80588e1c08 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.122719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1263/2501] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:07:54 2022] DELETE /identity/v3/projects/ea8fed976c5a45168625ccbbf99ab22b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:07:55.126667 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641aa0a-f28d-451a-afac-ad80588e1c08 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:55.129086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5641aa0a-f28d-451a-afac-ad80588e1c08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:55.171703 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c9b0b9-e023-4926-8362-b112720dead3 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:55.203682 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641aa0a-f28d-451a-afac-ad80588e1c08 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:55.214384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41271701-3cd9-46e9-984f-a95a2fdbca33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:55.215406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41271701-3cd9-46e9-984f-a95a2fdbca33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:55.215615 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41271701-3cd9-46e9-984f-a95a2fdbca33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:55.216509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41271701-3cd9-46e9-984f-a95a2fdbca33 admin admin] PATH_INFO: `/v3/users/c05662cd54bc4a26a2db024fedfeed37` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:55.217751 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41271701-3cd9-46e9-984f-a95a2fdbca33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c05662cd54bc4a26a2db024fedfeed37)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:55.219766 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41271701-3cd9-46e9-984f-a95a2fdbca33 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:55.231687 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:55.251707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e633ac-217d-4d5f-8183-ae6c9a0bed55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:55.252543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e633ac-217d-4d5f-8183-ae6c9a0bed55 service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:55.252960 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e633ac-217d-4d5f-8183-ae6c9a0bed55 service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:55.253310 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e633ac-217d-4d5f-8183-ae6c9a0bed55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:55.255661 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-70e633ac-217d-4d5f-8183-ae6c9a0bed55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:55.282913 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-41271701-3cd9-46e9-984f-a95a2fdbca33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c05662cd54bc4a26a2db024fedfeed37'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.292095 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-41271701-3cd9-46e9-984f-a95a2fdbca33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c05662cd54bc4a26a2db024fedfeed37'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.293012 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-41271701-3cd9-46e9-984f-a95a2fdbca33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c05662cd54bc4a26a2db024fedfeed37'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.307268 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-41271701-3cd9-46e9-984f-a95a2fdbca33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c05662cd54bc4a26a2db024fedfeed37'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.308293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-41271701-3cd9-46e9-984f-a95a2fdbca33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c05662cd54bc4a26a2db024fedfeed37'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.323183 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e633ac-217d-4d5f-8183-ae6c9a0bed55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:55.327581 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e633ac-217d-4d5f-8183-ae6c9a0bed55 service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:55.327715 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1264/2502] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:07:55 2022] DELETE /identity/v3/users/c05662cd54bc4a26a2db024fedfeed37 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:07:55.335645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1239/2503] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:07:55.567927 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41271701-3cd9-46e9-984f-a95a2fdbca33 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:55.569781 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-41271701-3cd9-46e9-984f-a95a2fdbca33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:55.674118 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41271701-3cd9-46e9-984f-a95a2fdbca33 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:55.681306 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07c2ad-b8da-482e-b4a7-b3ddf3707693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:55.682441 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07c2ad-b8da-482e-b4a7-b3ddf3707693 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:55.683372 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07c2ad-b8da-482e-b4a7-b3ddf3707693 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:55.683757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07c2ad-b8da-482e-b4a7-b3ddf3707693 admin admin] PATH_INFO: `/v3/users/230d5619188b4d27980be714b901c831` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:55.685699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e633ac-217d-4d5f-8183-ae6c9a0bed55 service nova] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:55.699417 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07c2ad-b8da-482e-b4a7-b3ddf3707693 admin admin] RBAC: Authorizing `identity:delete_user(user_id=230d5619188b4d27980be714b901c831)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:55.702072 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07c2ad-b8da-482e-b4a7-b3ddf3707693 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:55.702955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e633ac-217d-4d5f-8183-ae6c9a0bed55 service nova] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:55.714615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e84641-b593-4a9d-9a0c-1ca4cd102c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:55.715499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e84641-b593-4a9d-9a0c-1ca4cd102c97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:55.715784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e84641-b593-4a9d-9a0c-1ca4cd102c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:55.716062 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e84641-b593-4a9d-9a0c-1ca4cd102c97 admin admin] PATH_INFO: `/v3/projects/4375665e8d4644569083471a41b9dcc2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:55.719928 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e84641-b593-4a9d-9a0c-1ca4cd102c97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4375665e8d4644569083471a41b9dcc2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:55.721828 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e84641-b593-4a9d-9a0c-1ca4cd102c97 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:55.730023 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:55.789761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6c07c2ad-b8da-482e-b4a7-b3ddf3707693 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '230d5619188b4d27980be714b901c831'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.796993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-91e84641-b593-4a9d-9a0c-1ca4cd102c97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4375665e8d4644569083471a41b9dcc2', 'request_id': 'req-91e84641-b593-4a9d-9a0c-1ca4cd102c97'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.805190 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-91e84641-b593-4a9d-9a0c-1ca4cd102c97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4375665e8d4644569083471a41b9dcc2', 'request_id': 'req-91e84641-b593-4a9d-9a0c-1ca4cd102c97'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.808625 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6c07c2ad-b8da-482e-b4a7-b3ddf3707693 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '230d5619188b4d27980be714b901c831'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.810232 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6c07c2ad-b8da-482e-b4a7-b3ddf3707693 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '230d5619188b4d27980be714b901c831'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.814366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-91e84641-b593-4a9d-9a0c-1ca4cd102c97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4375665e8d4644569083471a41b9dcc2', 'request_id': 'req-91e84641-b593-4a9d-9a0c-1ca4cd102c97'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.833178 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-91e84641-b593-4a9d-9a0c-1ca4cd102c97 admin admin] The token cache is being invalidate because project 4375665e8d4644569083471a41b9dcc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:07:55.834339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-91e84641-b593-4a9d-9a0c-1ca4cd102c97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.835228 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6c07c2ad-b8da-482e-b4a7-b3ddf3707693 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '230d5619188b4d27980be714b901c831'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.835677 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-91e84641-b593-4a9d-9a0c-1ca4cd102c97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.836904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6c07c2ad-b8da-482e-b4a7-b3ddf3707693 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '230d5619188b4d27980be714b901c831'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:55.840393 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1240/2504] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:07:55 2022] DELETE /identity/v3/projects/4375665e8d4644569083471a41b9dcc2 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:07:55.845564 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5a5aa-3ce2-40d8-a0d5-de63c24e38fb None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:55.845921 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5a5aa-3ce2-40d8-a0d5-de63c24e38fb None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:55.846987 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa5a5aa-3ce2-40d8-a0d5-de63c24e38fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:55.871149 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1265/2505] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:07:55 2022] DELETE /identity/v3/users/230d5619188b4d27980be714b901c831 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:55.874479 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-afa5a5aa-3ce2-40d8-a0d5-de63c24e38fb None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:55.875837 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852c5d4-450c-447e-8e7f-be5ced8d5ac1 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:55.876203 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-afa5a5aa-3ce2-40d8-a0d5-de63c24e38fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:55.877089 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852c5d4-450c-447e-8e7f-be5ced8d5ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:55.878704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d852c5d4-450c-447e-8e7f-be5ced8d5ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:55.907645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-d852c5d4-450c-447e-8e7f-be5ced8d5ac1 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:55.909048 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-d852c5d4-450c-447e-8e7f-be5ced8d5ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:55.961097 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1241/2506] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:07:55.970035 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2947a5-cbfb-472f-8632-0364beba8ee0 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:55.970433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2947a5-cbfb-472f-8632-0364beba8ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:55.970882 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2947a5-cbfb-472f-8632-0364beba8ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:56.001793 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-3b2947a5-cbfb-472f-8632-0364beba8ee0 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:56.003584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-3b2947a5-cbfb-472f-8632-0364beba8ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:56.018652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1242/2507] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:56.021294 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2947a5-cbfb-472f-8632-0364beba8ee0 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:56.029324 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:56.038678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2947a5-cbfb-472f-8632-0364beba8ee0 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:56.048533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12fefe-7fd0-40a2-90b7-700993b2ece4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:56.049870 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12fefe-7fd0-40a2-90b7-700993b2ece4 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:56.050839 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12fefe-7fd0-40a2-90b7-700993b2ece4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:56.051215 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc12fefe-7fd0-40a2-90b7-700993b2ece4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:56.052696 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12fefe-7fd0-40a2-90b7-700993b2ece4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:07:56.053321 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12fefe-7fd0-40a2-90b7-700993b2ece4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:56.056496 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc12fefe-7fd0-40a2-90b7-700993b2ece4 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:56.068404 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1266/2508] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:07:56.074877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1243/2509] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:07:56.082279 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852c5d4-450c-447e-8e7f-be5ced8d5ac1 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:56.082504 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12fefe-7fd0-40a2-90b7-700993b2ece4 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:56.085279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-fc12fefe-7fd0-40a2-90b7-700993b2ece4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:56.097875 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d852c5d4-450c-447e-8e7f-be5ced8d5ac1 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:56.107096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167e178-b2af-424c-8c85-a06ef6df145d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:56.108066 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167e178-b2af-424c-8c85-a06ef6df145d admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:56.109161 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167e178-b2af-424c-8c85-a06ef6df145d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:56.109646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167e178-b2af-424c-8c85-a06ef6df145d admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:56.111630 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167e178-b2af-424c-8c85-a06ef6df145d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:56.116641 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167e178-b2af-424c-8c85-a06ef6df145d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:56.149169 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1267/2510] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:07:56 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:07:56.153778 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167e178-b2af-424c-8c85-a06ef6df145d admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:56.165643 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc12fefe-7fd0-40a2-90b7-700993b2ece4 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:56.222340 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167e178-b2af-424c-8c85-a06ef6df145d admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:56.225785 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c4ead-9eef-46df-8eb2-09184c178e6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:56.226451 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c4ead-9eef-46df-8eb2-09184c178e6f service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:56.227052 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6384cc-22b8-468a-8b01-324af533610a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:56.227315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c4ead-9eef-46df-8eb2-09184c178e6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:56.227605 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33c4ead-9eef-46df-8eb2-09184c178e6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:56.227744 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6384cc-22b8-468a-8b01-324af533610a admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:56.228030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6384cc-22b8-468a-8b01-324af533610a admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:56.228312 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6384cc-22b8-468a-8b01-324af533610a admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:56.229033 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6384cc-22b8-468a-8b01-324af533610a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:56.230987 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6384cc-22b8-468a-8b01-324af533610a admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:56.243659 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c4ead-9eef-46df-8eb2-09184c178e6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:56.245191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33c4ead-9eef-46df-8eb2-09184c178e6f service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:56.249719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:56.256763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1268/2511] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:07:56 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:07:56.256943 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1244/2512] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:07:56.260910 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6384cc-22b8-468a-8b01-324af533610a admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:56.272813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6384cc-22b8-468a-8b01-324af533610a admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:56.278128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c118024-2be7-4591-ac1f-942d990e850c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:56.278921 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c118024-2be7-4591-ac1f-942d990e850c admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:56.279284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c118024-2be7-4591-ac1f-942d990e850c admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:56.279635 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c118024-2be7-4591-ac1f-942d990e850c admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:56.280438 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c118024-2be7-4591-ac1f-942d990e850c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:56.282178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c118024-2be7-4591-ac1f-942d990e850c admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:56.296941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1269/2513] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:07:56.298159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c4ead-9eef-46df-8eb2-09184c178e6f service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:56.310946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33c4ead-9eef-46df-8eb2-09184c178e6f service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:56.316474 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e82f50-84cb-467c-aa43-dc653d7d7a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:56.317287 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e82f50-84cb-467c-aa43-dc653d7d7a28 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:56.317584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e82f50-84cb-467c-aa43-dc653d7d7a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:56.317868 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e82f50-84cb-467c-aa43-dc653d7d7a28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:56.318664 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e82f50-84cb-467c-aa43-dc653d7d7a28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:56.320498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e82f50-84cb-467c-aa43-dc653d7d7a28 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:56.334629 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1245/2514] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:07:56.338538 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c118024-2be7-4591-ac1f-942d990e850c admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:56.349649 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c118024-2be7-4591-ac1f-942d990e850c admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:56.354513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af1b3d-cb36-49ea-892d-52ddb95d148f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:56.355435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af1b3d-cb36-49ea-892d-52ddb95d148f admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:56.355719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af1b3d-cb36-49ea-892d-52ddb95d148f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:56.356301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33af1b3d-cb36-49ea-892d-52ddb95d148f admin admin] PATH_INFO: `/v3/projects/8696fb087d7944b0ada172af1b99a63b/users/36ebb8add21e423eb3a9abc6d7cd1717/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:56.374453 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af1b3d-cb36-49ea-892d-52ddb95d148f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8696fb087d7944b0ada172af1b99a63b, user_id=36ebb8add21e423eb3a9abc6d7cd1717, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:56.377170 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33af1b3d-cb36-49ea-892d-52ddb95d148f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:56.393675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1270/2515] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:07:56 2022] PUT /identity/v3/projects/8696fb087d7944b0ada172af1b99a63b/users/36ebb8add21e423eb3a9abc6d7cd1717/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:56.404103 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4b403-8239-4867-b471-e60c5b7d684b None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:56.404397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4b403-8239-4867-b471-e60c5b7d684b None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:56.404679 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f4b403-8239-4867-b471-e60c5b7d684b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:56.436186 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-f2f4b403-8239-4867-b471-e60c5b7d684b None None] MFA Rules not processed for user `36ebb8add21e423eb3a9abc6d7cd1717`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:56.437413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f2f4b403-8239-4867-b471-e60c5b7d684b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:56.526110 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1246/2516] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:56.530264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af1b3d-cb36-49ea-892d-52ddb95d148f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:56.566119 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4b403-8239-4867-b471-e60c5b7d684b None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:56.567687 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f2f4b403-8239-4867-b471-e60c5b7d684b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:56.628037 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33af1b3d-cb36-49ea-892d-52ddb95d148f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:56.632460 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:56.636074 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ccf40-eddf-4ef4-b9eb-4b7d9ceca589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:56.636931 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ccf40-eddf-4ef4-b9eb-4b7d9ceca589 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:56.637311 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ccf40-eddf-4ef4-b9eb-4b7d9ceca589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:56.637662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ccf40-eddf-4ef4-b9eb-4b7d9ceca589 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:56.638479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ccf40-eddf-4ef4-b9eb-4b7d9ceca589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:56.640580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ccf40-eddf-4ef4-b9eb-4b7d9ceca589 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:56.682157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1271/2517] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:07:56 2022] POST /identity/v3/projects => generated 385 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:07:56.683068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ccf40-eddf-4ef4-b9eb-4b7d9ceca589 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:56.684019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f4b403-8239-4867-b471-e60c5b7d684b None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:56.760755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee972a2-f3e3-406c-a6bf-e3351d588d83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:56.762773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee972a2-f3e3-406c-a6bf-e3351d588d83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:56.762773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee972a2-f3e3-406c-a6bf-e3351d588d83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:56.762773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee972a2-f3e3-406c-a6bf-e3351d588d83 admin admin] PATH_INFO: `/v3/projects/f84064db84a441a1a146641cf69efda1` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:56.763881 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee972a2-f3e3-406c-a6bf-e3351d588d83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f84064db84a441a1a146641cf69efda1)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:56.763995 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ccf40-eddf-4ef4-b9eb-4b7d9ceca589 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:56.766325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee972a2-f3e3-406c-a6bf-e3351d588d83 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:56.769495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ceacce-d877-46e3-87e1-bd65ae64353d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:56.770157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ceacce-d877-46e3-87e1-bd65ae64353d admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:56.770450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ceacce-d877-46e3-87e1-bd65ae64353d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:56.770765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ceacce-d877-46e3-87e1-bd65ae64353d admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:56.771454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ceacce-d877-46e3-87e1-bd65ae64353d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:56.773598 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ceacce-d877-46e3-87e1-bd65ae64353d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:56.803065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1272/2518] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:07:56 2022] POST /identity/v3/users => generated 299 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:07:56.807617 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ceacce-d877-46e3-87e1-bd65ae64353d admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:56.819796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ceacce-d877-46e3-87e1-bd65ae64353d admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:56.824598 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0ee972a2-f3e3-406c-a6bf-e3351d588d83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f84064db84a441a1a146641cf69efda1', 'request_id': 'req-0ee972a2-f3e3-406c-a6bf-e3351d588d83'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:56.831769 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0ee972a2-f3e3-406c-a6bf-e3351d588d83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f84064db84a441a1a146641cf69efda1', 'request_id': 'req-0ee972a2-f3e3-406c-a6bf-e3351d588d83'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:56.838383 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0ee972a2-f3e3-406c-a6bf-e3351d588d83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f84064db84a441a1a146641cf69efda1', 'request_id': 'req-0ee972a2-f3e3-406c-a6bf-e3351d588d83'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:56.855222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0ee972a2-f3e3-406c-a6bf-e3351d588d83 admin admin] The token cache is being invalidate because project f84064db84a441a1a146641cf69efda1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:07:56.855544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0ee972a2-f3e3-406c-a6bf-e3351d588d83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:56.856222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0ee972a2-f3e3-406c-a6bf-e3351d588d83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:56.857833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1247/2519] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:07:56 2022] DELETE /identity/v3/projects/f84064db84a441a1a146641cf69efda1 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:56.862648 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2d729-f98a-44c8-abd0-5cbbec50edff None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:56.862959 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2d729-f98a-44c8-abd0-5cbbec50edff None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:56.863244 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef2d729-f98a-44c8-abd0-5cbbec50edff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:56.879614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:56.884448 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-1ef2d729-f98a-44c8-abd0-5cbbec50edff None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:56.885688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-1ef2d729-f98a-44c8-abd0-5cbbec50edff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:56.910858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6d1e7-96cb-4a91-8216-4680ca6727fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:56.911558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6d1e7-96cb-4a91-8216-4680ca6727fd admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:56.911880 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6d1e7-96cb-4a91-8216-4680ca6727fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:56.912128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d6d1e7-96cb-4a91-8216-4680ca6727fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:56.912866 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d6d1e7-96cb-4a91-8216-4680ca6727fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:56.915829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80d6d1e7-96cb-4a91-8216-4680ca6727fd admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:56.927259 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1273/2520] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:56 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:07:56.932012 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6d1e7-96cb-4a91-8216-4680ca6727fd admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:56.933938 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-80d6d1e7-96cb-4a91-8216-4680ca6727fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:56.947788 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d6d1e7-96cb-4a91-8216-4680ca6727fd admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:56.952888 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba589341-af5b-4d08-9af6-ca8edff9dfb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:56.953900 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba589341-af5b-4d08-9af6-ca8edff9dfb1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:56.954292 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba589341-af5b-4d08-9af6-ca8edff9dfb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:56.954700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba589341-af5b-4d08-9af6-ca8edff9dfb1 admin admin] PATH_INFO: `/v3/projects/1eee72cbe2fb43c9a954197fcbec439c/users/efb6b1c9010b464cb054ca1d7243a3ff/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:56.965521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1248/2521] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:07:56.975032 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a72a8b-4706-48b8-87fe-82081f484aa8 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:56.975614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a72a8b-4706-48b8-87fe-82081f484aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:56.976715 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a72a8b-4706-48b8-87fe-82081f484aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:56.980487 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba589341-af5b-4d08-9af6-ca8edff9dfb1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eee72cbe2fb43c9a954197fcbec439c, user_id=efb6b1c9010b464cb054ca1d7243a3ff, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:56.983307 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba589341-af5b-4d08-9af6-ca8edff9dfb1 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:56.995753 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:57.003847 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-d6a72a8b-4706-48b8-87fe-82081f484aa8 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:57.005153 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1274/2522] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:07:56 2022] PUT /identity/v3/projects/1eee72cbe2fb43c9a954197fcbec439c/users/efb6b1c9010b464cb054ca1d7243a3ff/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:07:57.008316 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d6a72a8b-4706-48b8-87fe-82081f484aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:57.009720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba589341-af5b-4d08-9af6-ca8edff9dfb1 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:57.088511 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1249/2523] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:57.093674 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a72a8b-4706-48b8-87fe-82081f484aa8 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:57.104295 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba589341-af5b-4d08-9af6-ca8edff9dfb1 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:57.108469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a72a8b-4706-48b8-87fe-82081f484aa8 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:57.110425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73475d27-5e79-4d05-806b-1672342b3a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:57.111585 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73475d27-5e79-4d05-806b-1672342b3a7f admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:57.112126 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73475d27-5e79-4d05-806b-1672342b3a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:57.112665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73475d27-5e79-4d05-806b-1672342b3a7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:57.113554 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73475d27-5e79-4d05-806b-1672342b3a7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:57.113835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee46058-522a-40ba-8b2e-c7bebdd5b5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:57.114507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee46058-522a-40ba-8b2e-c7bebdd5b5dd admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:57.114842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee46058-522a-40ba-8b2e-c7bebdd5b5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:57.115120 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee46058-522a-40ba-8b2e-c7bebdd5b5dd admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:57.116030 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee46058-522a-40ba-8b2e-c7bebdd5b5dd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:07:57.116223 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73475d27-5e79-4d05-806b-1672342b3a7f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:57.116347 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee46058-522a-40ba-8b2e-c7bebdd5b5dd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:57.118159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee46058-522a-40ba-8b2e-c7bebdd5b5dd admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:57.128853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1275/2524] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:07:57.134293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1250/2525] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:07:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:07:57.134608 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73475d27-5e79-4d05-806b-1672342b3a7f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:57.138222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee46058-522a-40ba-8b2e-c7bebdd5b5dd admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:57.147848 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73475d27-5e79-4d05-806b-1672342b3a7f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:57.151565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee46058-522a-40ba-8b2e-c7bebdd5b5dd admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:57.153824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb9e39a-e6c8-4f9a-9aab-00a8d73d63dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:57.154802 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9e39a-e6c8-4f9a-9aab-00a8d73d63dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:57.155092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9e39a-e6c8-4f9a-9aab-00a8d73d63dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:57.155375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb9e39a-e6c8-4f9a-9aab-00a8d73d63dd admin admin] PATH_INFO: `/v3/projects/1eee72cbe2fb43c9a954197fcbec439c/users/efb6b1c9010b464cb054ca1d7243a3ff/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:57.157487 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9e39a-e6c8-4f9a-9aab-00a8d73d63dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eee72cbe2fb43c9a954197fcbec439c, user_id=efb6b1c9010b464cb054ca1d7243a3ff, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:57.158218 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f47c5-bac0-4c91-87e2-8f9d330a0096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:57.159038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f47c5-bac0-4c91-87e2-8f9d330a0096 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:57.159325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f47c5-bac0-4c91-87e2-8f9d330a0096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:57.162304 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb9e39a-e6c8-4f9a-9aab-00a8d73d63dd admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:57.163421 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72f47c5-bac0-4c91-87e2-8f9d330a0096 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:57.164153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f47c5-bac0-4c91-87e2-8f9d330a0096 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:57.168440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72f47c5-bac0-4c91-87e2-8f9d330a0096 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:57.182911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1276/2526] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:07:57 2022] PUT /identity/v3/projects/1eee72cbe2fb43c9a954197fcbec439c/users/efb6b1c9010b464cb054ca1d7243a3ff/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:57.193276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e0d63-e073-4506-9de0-8382345483bf None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:57.193616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e0d63-e073-4506-9de0-8382345483bf None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:57.194012 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942e0d63-e073-4506-9de0-8382345483bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:57.205298 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1251/2527] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:07:57 2022] POST /identity/v3/projects => generated 367 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:07:57.217424 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f47c5-bac0-4c91-87e2-8f9d330a0096 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:57.231637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-942e0d63-e073-4506-9de0-8382345483bf None None] MFA Rules not processed for user `efb6b1c9010b464cb054ca1d7243a3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:57.232789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-942e0d63-e073-4506-9de0-8382345483bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:57.298821 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72f47c5-bac0-4c91-87e2-8f9d330a0096 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:57.304155 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a3e9b-6580-4209-9850-46a056803a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:57.308964 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a3e9b-6580-4209-9850-46a056803a89 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:57.309146 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1277/2528] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:57 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:57.309250 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a3e9b-6580-4209-9850-46a056803a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:57.309250 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98a3e9b-6580-4209-9850-46a056803a89 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:57.309250 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a3e9b-6580-4209-9850-46a056803a89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:57.309250 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98a3e9b-6580-4209-9850-46a056803a89 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:57.316236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e0d63-e073-4506-9de0-8382345483bf None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:57.317606 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-942e0d63-e073-4506-9de0-8382345483bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:57.337614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1252/2529] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:07:57 2022] POST /identity/v3/users => generated 284 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:07:57.341635 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a3e9b-6580-4209-9850-46a056803a89 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:57.345900 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:57.351963 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98a3e9b-6580-4209-9850-46a056803a89 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:57.358225 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605ea26-4205-4519-a668-76143bdf8741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:57.358956 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605ea26-4205-4519-a668-76143bdf8741 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:57.359241 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605ea26-4205-4519-a668-76143bdf8741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:57.359518 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8605ea26-4205-4519-a668-76143bdf8741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:57.360243 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605ea26-4205-4519-a668-76143bdf8741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:57.361846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8605ea26-4205-4519-a668-76143bdf8741 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:57.372504 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1253/2530] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:07:57.377167 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605ea26-4205-4519-a668-76143bdf8741 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:57.387767 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8605ea26-4205-4519-a668-76143bdf8741 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:57.392928 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b72223-e95f-4d01-857f-d722f93d2f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:57.393565 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942e0d63-e073-4506-9de0-8382345483bf None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:57.393684 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b72223-e95f-4d01-857f-d722f93d2f13 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:57.393874 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b72223-e95f-4d01-857f-d722f93d2f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:57.394162 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b72223-e95f-4d01-857f-d722f93d2f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:57.394921 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b72223-e95f-4d01-857f-d722f93d2f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:57.396516 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b72223-e95f-4d01-857f-d722f93d2f13 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:57.397911 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68040dbb-e87a-48ec-af9f-47d0f86b102e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:57.398707 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68040dbb-e87a-48ec-af9f-47d0f86b102e service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:57.399065 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68040dbb-e87a-48ec-af9f-47d0f86b102e service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:57.400032 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68040dbb-e87a-48ec-af9f-47d0f86b102e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:57.408269 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1254/2531] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:07:57.411073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68040dbb-e87a-48ec-af9f-47d0f86b102e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:57.412301 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b72223-e95f-4d01-857f-d722f93d2f13 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:57.413156 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68040dbb-e87a-48ec-af9f-47d0f86b102e service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:57.425186 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b72223-e95f-4d01-857f-d722f93d2f13 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:57.431803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b54100-533f-47ea-bf04-02c13504320f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:57.432717 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b54100-533f-47ea-bf04-02c13504320f admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:57.433052 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b54100-533f-47ea-bf04-02c13504320f admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:57.434727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b54100-533f-47ea-bf04-02c13504320f admin admin] PATH_INFO: `/v3/projects/c5dcade3e63a44e6a998e8949ccce206/users/52c28aec93f8459e9d3216672c759b42/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:57.435142 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1278/2532] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:07:57.452142 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b54100-533f-47ea-bf04-02c13504320f admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5dcade3e63a44e6a998e8949ccce206, user_id=52c28aec93f8459e9d3216672c759b42, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:57.456140 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b54100-533f-47ea-bf04-02c13504320f admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:57.471558 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1255/2533] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:07:57 2022] PUT /identity/v3/projects/c5dcade3e63a44e6a998e8949ccce206/users/52c28aec93f8459e9d3216672c759b42/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:07:57.482974 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2b6ea-6f7c-4340-a13b-0257500e2084 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:57.483247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2b6ea-6f7c-4340-a13b-0257500e2084 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:57.483462 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2b6ea-6f7c-4340-a13b-0257500e2084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:57.504403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b54100-533f-47ea-bf04-02c13504320f admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:57.520933 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-88b2b6ea-6f7c-4340-a13b-0257500e2084 None None] MFA Rules not processed for user `52c28aec93f8459e9d3216672c759b42`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:57.522277 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-88b2b6ea-6f7c-4340-a13b-0257500e2084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:57.526665 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:57.588917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b54100-533f-47ea-bf04-02c13504320f admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:57.593886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1312bab1-ba6d-416c-a4fb-15bf3e22381c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:57.594824 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312bab1-ba6d-416c-a4fb-15bf3e22381c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:57.595115 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312bab1-ba6d-416c-a4fb-15bf3e22381c admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:57.595396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1312bab1-ba6d-416c-a4fb-15bf3e22381c admin admin] PATH_INFO: `/v3/users/36ebb8add21e423eb3a9abc6d7cd1717` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:57.596640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312bab1-ba6d-416c-a4fb-15bf3e22381c admin admin] RBAC: Authorizing `identity:delete_user(user_id=36ebb8add21e423eb3a9abc6d7cd1717)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:57.598778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1312bab1-ba6d-416c-a4fb-15bf3e22381c admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:57.608099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1279/2534] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:57 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:57.616223 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2b6ea-6f7c-4340-a13b-0257500e2084 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:57.638618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2b6ea-6f7c-4340-a13b-0257500e2084 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:57.644416 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d4d6c-3b44-4925-944c-4a0cf0b93dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:57.645092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d4d6c-3b44-4925-944c-4a0cf0b93dfb admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:57.645376 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d4d6c-3b44-4925-944c-4a0cf0b93dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:57.645656 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d4d6c-3b44-4925-944c-4a0cf0b93dfb admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:57.646476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d4d6c-3b44-4925-944c-4a0cf0b93dfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:57.648425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d4d6c-3b44-4925-944c-4a0cf0b93dfb admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:57.665790 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1312bab1-ba6d-416c-a4fb-15bf3e22381c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '36ebb8add21e423eb3a9abc6d7cd1717'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:57.666829 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1312bab1-ba6d-416c-a4fb-15bf3e22381c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '36ebb8add21e423eb3a9abc6d7cd1717'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:57.676520 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1312bab1-ba6d-416c-a4fb-15bf3e22381c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '36ebb8add21e423eb3a9abc6d7cd1717'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:57.677307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1312bab1-ba6d-416c-a4fb-15bf3e22381c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '36ebb8add21e423eb3a9abc6d7cd1717'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:57.686949 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1280/2535] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:07:57 2022] POST /identity/v3/projects => generated 367 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:07:57.690978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d4d6c-3b44-4925-944c-4a0cf0b93dfb admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:57.694658 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-bf5d4d6c-3b44-4925-944c-4a0cf0b93dfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:57.695009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1312bab1-ba6d-416c-a4fb-15bf3e22381c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '36ebb8add21e423eb3a9abc6d7cd1717'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:57.711798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1256/2536] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:07:57 2022] DELETE /identity/v3/users/36ebb8add21e423eb3a9abc6d7cd1717 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:57.719480 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f67ff3-6206-4c1f-aa85-603f57067134 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:57.719772 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f67ff3-6206-4c1f-aa85-603f57067134 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:57.720053 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f67ff3-6206-4c1f-aa85-603f57067134 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:57.739059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-92f67ff3-6206-4c1f-aa85-603f57067134 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:57.740174 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-92f67ff3-6206-4c1f-aa85-603f57067134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:57.788981 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d4d6c-3b44-4925-944c-4a0cf0b93dfb admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:57.824675 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1257/2537] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:57.836238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f67ff3-6206-4c1f-aa85-603f57067134 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:57.838622 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-92f67ff3-6206-4c1f-aa85-603f57067134 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:57.844804 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:57.882236 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b3bf8-73cf-4773-8204-c1f986bcca01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:57.883263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b3bf8-73cf-4773-8204-c1f986bcca01 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:57.883762 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b3bf8-73cf-4773-8204-c1f986bcca01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:57.885546 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643b3bf8-73cf-4773-8204-c1f986bcca01 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:57.886584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b3bf8-73cf-4773-8204-c1f986bcca01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:57.889536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643b3bf8-73cf-4773-8204-c1f986bcca01 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:57.904736 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:57.918053 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1281/2538] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:07:57 2022] POST /identity/v3/users => generated 290 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:07:57.922055 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b3bf8-73cf-4773-8204-c1f986bcca01 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:57.933125 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643b3bf8-73cf-4773-8204-c1f986bcca01 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:57.938773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50da86-36c6-4080-87c4-302f6203cd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:57.939463 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50da86-36c6-4080-87c4-302f6203cd73 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:57.939753 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50da86-36c6-4080-87c4-302f6203cd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:57.939989 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f67ff3-6206-4c1f-aa85-603f57067134 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:57.940120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d50da86-36c6-4080-87c4-302f6203cd73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:57.941038 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50da86-36c6-4080-87c4-302f6203cd73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:57.942794 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d50da86-36c6-4080-87c4-302f6203cd73 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:57.947639 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d7ff2-feb2-4de5-b810-2d380960f080 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:57.949288 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d7ff2-feb2-4de5-b810-2d380960f080 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:57.949584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d7ff2-feb2-4de5-b810-2d380960f080 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:57.949867 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69d7ff2-feb2-4de5-b810-2d380960f080 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:57.958410 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1282/2539] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:07:57.967699 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50da86-36c6-4080-87c4-302f6203cd73 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:57.970304 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d7ff2-feb2-4de5-b810-2d380960f080 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:57.971963 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69d7ff2-feb2-4de5-b810-2d380960f080 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:57.981430 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d50da86-36c6-4080-87c4-302f6203cd73 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:57.988174 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1258/2540] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:07:57.992244 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf3460-3583-4572-8275-cdc0fe2212cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:57.994226 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf3460-3583-4572-8275-cdc0fe2212cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:57.994715 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf3460-3583-4572-8275-cdc0fe2212cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:57.995432 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbf3460-3583-4572-8275-cdc0fe2212cf admin admin] PATH_INFO: `/v3/projects/8b3aeb2c19014b43bdcd045b1a3991db/users/3f86e68ff70d4067a8e2ea3a60d91336/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:58.027712 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf3460-3583-4572-8275-cdc0fe2212cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b3aeb2c19014b43bdcd045b1a3991db, user_id=3f86e68ff70d4067a8e2ea3a60d91336, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:58.030730 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbf3460-3583-4572-8275-cdc0fe2212cf admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:58.051217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1283/2541] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:07:57 2022] PUT /identity/v3/projects/8b3aeb2c19014b43bdcd045b1a3991db/users/3f86e68ff70d4067a8e2ea3a60d91336/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:58.055489 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d7ff2-feb2-4de5-b810-2d380960f080 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:58.067447 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69d7ff2-feb2-4de5-b810-2d380960f080 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:58.157716 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebbedb-2032-4538-8ae5-1086a49eece3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:58.158571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebbedb-2032-4538-8ae5-1086a49eece3 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:58.158955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebbedb-2032-4538-8ae5-1086a49eece3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:58.159245 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebbedb-2032-4538-8ae5-1086a49eece3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:58.160027 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebbedb-2032-4538-8ae5-1086a49eece3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:58.161876 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebbedb-2032-4538-8ae5-1086a49eece3 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:58.206827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1259/2542] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:07:58 2022] GET /identity/v3/roles => generated 1420 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:07:58.214407 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf3460-3583-4572-8275-cdc0fe2212cf admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:58.225659 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbf3460-3583-4572-8275-cdc0fe2212cf admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:58.238110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2169e5e-5aa8-4182-8288-4f0f271ca54d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:58.239091 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2169e5e-5aa8-4182-8288-4f0f271ca54d admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:58.239383 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2169e5e-5aa8-4182-8288-4f0f271ca54d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:58.239666 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2169e5e-5aa8-4182-8288-4f0f271ca54d admin admin] PATH_INFO: `/v3/projects/8b3aeb2c19014b43bdcd045b1a3991db/users/3f86e68ff70d4067a8e2ea3a60d91336/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:58.241769 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2169e5e-5aa8-4182-8288-4f0f271ca54d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b3aeb2c19014b43bdcd045b1a3991db, user_id=3f86e68ff70d4067a8e2ea3a60d91336, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:58.245241 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2169e5e-5aa8-4182-8288-4f0f271ca54d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:58.258165 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1284/2543] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:07:58 2022] PUT /identity/v3/projects/8b3aeb2c19014b43bdcd045b1a3991db/users/3f86e68ff70d4067a8e2ea3a60d91336/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:07:58.268238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a6269-3877-4d0b-9df6-49be8720a490 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:58.268535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a6269-3877-4d0b-9df6-49be8720a490 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:58.268822 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a6269-3877-4d0b-9df6-49be8720a490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:58.309059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-1e3a6269-3877-4d0b-9df6-49be8720a490 None None] MFA Rules not processed for user `3f86e68ff70d4067a8e2ea3a60d91336`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:58.311003 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-1e3a6269-3877-4d0b-9df6-49be8720a490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:58.420959 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1260/2544] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:58 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:58.441922 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2169e5e-5aa8-4182-8288-4f0f271ca54d admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:58.444215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-d2169e5e-5aa8-4182-8288-4f0f271ca54d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:58.521811 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2169e5e-5aa8-4182-8288-4f0f271ca54d admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:58.525904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c547b36-bb4f-4daf-bc8e-5af9e1c12c17 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:58.526663 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c547b36-bb4f-4daf-bc8e-5af9e1c12c17 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:58.527031 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c547b36-bb4f-4daf-bc8e-5af9e1c12c17 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:58.527388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c547b36-bb4f-4daf-bc8e-5af9e1c12c17 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:58.537053 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c547b36-bb4f-4daf-bc8e-5af9e1c12c17 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:58.538492 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c547b36-bb4f-4daf-bc8e-5af9e1c12c17 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:58.549555 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1285/2545] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:07:58 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:07:58.746643 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a6269-3877-4d0b-9df6-49be8720a490 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:58.748852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-1e3a6269-3877-4d0b-9df6-49be8720a490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:58.839739 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c547b36-bb4f-4daf-bc8e-5af9e1c12c17 service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:58.841650 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-9c547b36-bb4f-4daf-bc8e-5af9e1c12c17 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:58.864185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a6269-3877-4d0b-9df6-49be8720a490 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:58.870284 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb777d2d-a28e-49eb-8e58-df71d8e77c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:58.871309 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb777d2d-a28e-49eb-8e58-df71d8e77c7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:58.871621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb777d2d-a28e-49eb-8e58-df71d8e77c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:58.871899 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb777d2d-a28e-49eb-8e58-df71d8e77c7c admin admin] PATH_INFO: `/v3/projects/8696fb087d7944b0ada172af1b99a63b` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:58.873674 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb777d2d-a28e-49eb-8e58-df71d8e77c7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=8696fb087d7944b0ada172af1b99a63b)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:58.875623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb777d2d-a28e-49eb-8e58-df71d8e77c7c admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:58.878376 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:58.925211 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c547b36-bb4f-4daf-bc8e-5af9e1c12c17 service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:58.938009 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-fb777d2d-a28e-49eb-8e58-df71d8e77c7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8696fb087d7944b0ada172af1b99a63b', 'request_id': 'req-fb777d2d-a28e-49eb-8e58-df71d8e77c7c'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:58.945350 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-fb777d2d-a28e-49eb-8e58-df71d8e77c7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8696fb087d7944b0ada172af1b99a63b', 'request_id': 'req-fb777d2d-a28e-49eb-8e58-df71d8e77c7c'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:58.952046 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-fb777d2d-a28e-49eb-8e58-df71d8e77c7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8696fb087d7944b0ada172af1b99a63b', 'request_id': 'req-fb777d2d-a28e-49eb-8e58-df71d8e77c7c'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:58.954164 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:58.967678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-fb777d2d-a28e-49eb-8e58-df71d8e77c7c admin admin] The token cache is being invalidate because project 8696fb087d7944b0ada172af1b99a63b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:07:58.968039 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-fb777d2d-a28e-49eb-8e58-df71d8e77c7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:58.970316 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-fb777d2d-a28e-49eb-8e58-df71d8e77c7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:58.974981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1261/2546] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:07:58 2022] DELETE /identity/v3/projects/8696fb087d7944b0ada172af1b99a63b => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:58.976307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb777d2d-a28e-49eb-8e58-df71d8e77c7c admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:58.978564 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-fb777d2d-a28e-49eb-8e58-df71d8e77c7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:58.998896 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0dadc-d871-4efc-9673-a417b68f5047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:58.998896 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0dadc-d871-4efc-9673-a417b68f5047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:58.998896 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0dadc-d871-4efc-9673-a417b68f5047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:58.998896 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f0dadc-d871-4efc-9673-a417b68f5047 admin admin] PATH_INFO: `/v3/users/a1b1a557a3834b98a3397c41665b5dc6` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:58.999287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0dadc-d871-4efc-9673-a417b68f5047 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1b1a557a3834b98a3397c41665b5dc6)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:59.000771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f0dadc-d871-4efc-9673-a417b68f5047 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:59.052195 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb777d2d-a28e-49eb-8e58-df71d8e77c7c admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:59.057348 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd543d1-ca20-4b96-869c-9fa1ac4035cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:59.058366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd543d1-ca20-4b96-869c-9fa1ac4035cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:59.058787 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd543d1-ca20-4b96-869c-9fa1ac4035cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:59.059128 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd543d1-ca20-4b96-869c-9fa1ac4035cd admin admin] PATH_INFO: `/v3/users/efb6b1c9010b464cb054ca1d7243a3ff` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:59.059439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-14f0dadc-d871-4efc-9673-a417b68f5047 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1b1a557a3834b98a3397c41665b5dc6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.060774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd543d1-ca20-4b96-869c-9fa1ac4035cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=efb6b1c9010b464cb054ca1d7243a3ff)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:59.063188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd543d1-ca20-4b96-869c-9fa1ac4035cd admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:59.068463 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-14f0dadc-d871-4efc-9673-a417b68f5047 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1b1a557a3834b98a3397c41665b5dc6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.069173 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-14f0dadc-d871-4efc-9673-a417b68f5047 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1b1a557a3834b98a3397c41665b5dc6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.084736 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-14f0dadc-d871-4efc-9673-a417b68f5047 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1b1a557a3834b98a3397c41665b5dc6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.086421 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-14f0dadc-d871-4efc-9673-a417b68f5047 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1b1a557a3834b98a3397c41665b5dc6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.105005 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1286/2547] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:07:58 2022] DELETE /identity/v3/users/a1b1a557a3834b98a3397c41665b5dc6 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:07:59.127385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2bd543d1-ca20-4b96-869c-9fa1ac4035cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efb6b1c9010b464cb054ca1d7243a3ff'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.128354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2bd543d1-ca20-4b96-869c-9fa1ac4035cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efb6b1c9010b464cb054ca1d7243a3ff'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.138224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2bd543d1-ca20-4b96-869c-9fa1ac4035cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efb6b1c9010b464cb054ca1d7243a3ff'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.143081 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2bd543d1-ca20-4b96-869c-9fa1ac4035cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efb6b1c9010b464cb054ca1d7243a3ff'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.163629 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2bd543d1-ca20-4b96-869c-9fa1ac4035cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efb6b1c9010b464cb054ca1d7243a3ff'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.181863 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1262/2548] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:07:58 2022] DELETE /identity/v3/users/efb6b1c9010b464cb054ca1d7243a3ff => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:07:59.338374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0dadc-d871-4efc-9673-a417b68f5047 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:59.340206 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-14f0dadc-d871-4efc-9673-a417b68f5047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:59.498417 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f0dadc-d871-4efc-9673-a417b68f5047 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:59.596927 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:59.609948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8579a69e-0275-4a3e-a782-29e3cc3a44a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:59.611584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579a69e-0275-4a3e-a782-29e3cc3a44a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:59.611584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579a69e-0275-4a3e-a782-29e3cc3a44a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:59.612028 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8579a69e-0275-4a3e-a782-29e3cc3a44a6 admin admin] PATH_INFO: `/v3/projects/6fc0acf708204e1f8f4510cdb1bbf442` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:59.613345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8579a69e-0275-4a3e-a782-29e3cc3a44a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fc0acf708204e1f8f4510cdb1bbf442)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:59.616171 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8579a69e-0275-4a3e-a782-29e3cc3a44a6 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:59.633229 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd543d1-ca20-4b96-869c-9fa1ac4035cd admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:59.635452 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-2bd543d1-ca20-4b96-869c-9fa1ac4035cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:59.656345 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd543d1-ca20-4b96-869c-9fa1ac4035cd admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:07:59.662442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd4667c-5677-4cf1-b9d2-a058a1465b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:07:59.663643 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4667c-5677-4cf1-b9d2-a058a1465b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:59.664191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4667c-5677-4cf1-b9d2-a058a1465b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:59.664493 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd4667c-5677-4cf1-b9d2-a058a1465b2a admin admin] PATH_INFO: `/v3/projects/1eee72cbe2fb43c9a954197fcbec439c` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:59.666299 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd4667c-5677-4cf1-b9d2-a058a1465b2a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eee72cbe2fb43c9a954197fcbec439c)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:07:59.668272 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd4667c-5677-4cf1-b9d2-a058a1465b2a admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:07:59.671032 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:07:59.725858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8579a69e-0275-4a3e-a782-29e3cc3a44a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fc0acf708204e1f8f4510cdb1bbf442', 'request_id': 'req-8579a69e-0275-4a3e-a782-29e3cc3a44a6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.736552 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cbd4667c-5677-4cf1-b9d2-a058a1465b2a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eee72cbe2fb43c9a954197fcbec439c', 'request_id': 'req-cbd4667c-5677-4cf1-b9d2-a058a1465b2a'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.736667 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8579a69e-0275-4a3e-a782-29e3cc3a44a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fc0acf708204e1f8f4510cdb1bbf442', 'request_id': 'req-8579a69e-0275-4a3e-a782-29e3cc3a44a6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.743045 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8579a69e-0275-4a3e-a782-29e3cc3a44a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fc0acf708204e1f8f4510cdb1bbf442', 'request_id': 'req-8579a69e-0275-4a3e-a782-29e3cc3a44a6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.745874 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cbd4667c-5677-4cf1-b9d2-a058a1465b2a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eee72cbe2fb43c9a954197fcbec439c', 'request_id': 'req-cbd4667c-5677-4cf1-b9d2-a058a1465b2a'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.753611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cbd4667c-5677-4cf1-b9d2-a058a1465b2a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eee72cbe2fb43c9a954197fcbec439c', 'request_id': 'req-cbd4667c-5677-4cf1-b9d2-a058a1465b2a'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.788868 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cbd4667c-5677-4cf1-b9d2-a058a1465b2a admin admin] The token cache is being invalidate because project 1eee72cbe2fb43c9a954197fcbec439c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:07:59.789394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8579a69e-0275-4a3e-a782-29e3cc3a44a6 admin admin] The token cache is being invalidate because project 6fc0acf708204e1f8f4510cdb1bbf442 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:07:59.789720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8579a69e-0275-4a3e-a782-29e3cc3a44a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.790455 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8579a69e-0275-4a3e-a782-29e3cc3a44a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.791249 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cbd4667c-5677-4cf1-b9d2-a058a1465b2a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.792081 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cbd4667c-5677-4cf1-b9d2-a058a1465b2a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:07:59.795437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1287/2549] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:07:59 2022] DELETE /identity/v3/projects/6fc0acf708204e1f8f4510cdb1bbf442 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:07:59.796858 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12562e2f-e39c-4c39-a1da-a69fe0f40f4d None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:59.797154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12562e2f-e39c-4c39-a1da-a69fe0f40f4d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:59.797437 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12562e2f-e39c-4c39-a1da-a69fe0f40f4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:59.803173 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1263/2550] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:07:59 2022] DELETE /identity/v3/projects/1eee72cbe2fb43c9a954197fcbec439c => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:07:59.804883 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6d2b3-419d-4000-84ea-d9c252fcf669 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:59.805179 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6d2b3-419d-4000-84ea-d9c252fcf669 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:59.805500 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d6d2b3-419d-4000-84ea-d9c252fcf669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:59.847050 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-12562e2f-e39c-4c39-a1da-a69fe0f40f4d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:59.849086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-12562e2f-e39c-4c39-a1da-a69fe0f40f4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:59.873817 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-00d6d2b3-419d-4000-84ea-d9c252fcf669 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:59.875214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-00d6d2b3-419d-4000-84ea-d9c252fcf669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:59.932974 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1288/2551] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:07:59.941697 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136bbb1f-ae4d-4148-937c-6b07429e0752 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:59.942536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136bbb1f-ae4d-4148-937c-6b07429e0752 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:59.942895 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-136bbb1f-ae4d-4148-937c-6b07429e0752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:59.963409 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-136bbb1f-ae4d-4148-937c-6b07429e0752 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:07:59.965010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-136bbb1f-ae4d-4148-937c-6b07429e0752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:07:59.975736 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1289/2552] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:07:59.979725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136bbb1f-ae4d-4148-937c-6b07429e0752 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:07:59.981215 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1264/2553] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:07:59.989599 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6d27d-bffb-4c00-a06e-53c3daba4783 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:07:59.989892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6d27d-bffb-4c00-a06e-53c3daba4783 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:07:59.990172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec6d27d-bffb-4c00-a06e-53c3daba4783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:07:59.995052 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-136bbb1f-ae4d-4148-937c-6b07429e0752 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.000982 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1741b55-d9d1-445b-9a22-d78bba19d142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.001851 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1741b55-d9d1-445b-9a22-d78bba19d142 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.002212 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1741b55-d9d1-445b-9a22-d78bba19d142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.002559 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1741b55-d9d1-445b-9a22-d78bba19d142 admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.003822 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1741b55-d9d1-445b-9a22-d78bba19d142 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:08:00.004138 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1741b55-d9d1-445b-9a22-d78bba19d142 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.006010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1741b55-d9d1-445b-9a22-d78bba19d142 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.010295 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-9ec6d27d-bffb-4c00-a06e-53c3daba4783 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:00.011750 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-9ec6d27d-bffb-4c00-a06e-53c3daba4783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:00.021611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1265/2554] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:07:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:00.021859 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1290/2555] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 23:08:00.026133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6d27d-bffb-4c00-a06e-53c3daba4783 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:00.027486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1741b55-d9d1-445b-9a22-d78bba19d142 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:00.037774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec6d27d-bffb-4c00-a06e-53c3daba4783 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.038452 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1741b55-d9d1-445b-9a22-d78bba19d142 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.043054 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91087d5c-268e-4397-a81d-e2cd07827b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.043732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91087d5c-268e-4397-a81d-e2cd07827b9a admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.044019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91087d5c-268e-4397-a81d-e2cd07827b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.044316 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91087d5c-268e-4397-a81d-e2cd07827b9a admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.045034 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91087d5c-268e-4397-a81d-e2cd07827b9a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:08:00.045349 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91087d5c-268e-4397-a81d-e2cd07827b9a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.046044 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c16df-5214-4004-b514-59ac5d16a6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.046749 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c16df-5214-4004-b514-59ac5d16a6fc admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.047024 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91087d5c-268e-4397-a81d-e2cd07827b9a admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.047149 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c16df-5214-4004-b514-59ac5d16a6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.047866 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80c16df-5214-4004-b514-59ac5d16a6fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.048144 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c16df-5214-4004-b514-59ac5d16a6fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.050052 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80c16df-5214-4004-b514-59ac5d16a6fc admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.059115 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1266/2556] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 23:08:00.063320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91087d5c-268e-4397-a81d-e2cd07827b9a admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:00.073955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91087d5c-268e-4397-a81d-e2cd07827b9a admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.079306 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633f91e-2b87-4892-83d4-9d5738a16992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.079469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1291/2557] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:00 2022] POST /identity/v3/projects => generated 405 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:00.082612 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633f91e-2b87-4892-83d4-9d5738a16992 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.082612 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633f91e-2b87-4892-83d4-9d5738a16992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.082612 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e633f91e-2b87-4892-83d4-9d5738a16992 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.082612 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633f91e-2b87-4892-83d4-9d5738a16992 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.084030 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e633f91e-2b87-4892-83d4-9d5738a16992 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.084350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c16df-5214-4004-b514-59ac5d16a6fc admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:00.116933 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1267/2558] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:00 2022] POST /identity/v3/projects => generated 401 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:00.118409 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633f91e-2b87-4892-83d4-9d5738a16992 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:00.149704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:00.178902 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80c16df-5214-4004-b514-59ac5d16a6fc admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.192885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:00.208998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e633f91e-2b87-4892-83d4-9d5738a16992 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.217683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d38a2e-ea8f-4715-9e4d-2bc6f42b3cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.218851 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d38a2e-ea8f-4715-9e4d-2bc6f42b3cc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.219146 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d38a2e-ea8f-4715-9e4d-2bc6f42b3cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.219432 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d38a2e-ea8f-4715-9e4d-2bc6f42b3cc3 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.220126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d38a2e-ea8f-4715-9e4d-2bc6f42b3cc3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.224154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d38a2e-ea8f-4715-9e4d-2bc6f42b3cc3 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.252907 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707f50b-c449-49ab-be97-5797fedad4d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.253892 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707f50b-c449-49ab-be97-5797fedad4d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.254177 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707f50b-c449-49ab-be97-5797fedad4d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.255116 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d707f50b-c449-49ab-be97-5797fedad4d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.255824 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707f50b-c449-49ab-be97-5797fedad4d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.257396 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1268/2559] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:00 2022] POST /identity/v3/users => generated 301 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:00.257985 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d707f50b-c449-49ab-be97-5797fedad4d9 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.264287 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d38a2e-ea8f-4715-9e4d-2bc6f42b3cc3 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:00.274736 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d38a2e-ea8f-4715-9e4d-2bc6f42b3cc3 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.279975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31e5c9-584a-4c91-b0c7-a2187618250b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.283307 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1292/2560] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:00 2022] POST /identity/v3/users => generated 303 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:00.283435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31e5c9-584a-4c91-b0c7-a2187618250b admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.283435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31e5c9-584a-4c91-b0c7-a2187618250b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.283435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d31e5c9-584a-4c91-b0c7-a2187618250b admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.283435 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31e5c9-584a-4c91-b0c7-a2187618250b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.283931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d31e5c9-584a-4c91-b0c7-a2187618250b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.284552 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707f50b-c449-49ab-be97-5797fedad4d9 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:00.294704 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1269/2561] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:00.296225 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d707f50b-c449-49ab-be97-5797fedad4d9 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.298343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31e5c9-584a-4c91-b0c7-a2187618250b admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:00.302125 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56399c8d-6089-42de-ba74-9b4104f1879b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.302860 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56399c8d-6089-42de-ba74-9b4104f1879b admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.303152 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56399c8d-6089-42de-ba74-9b4104f1879b admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.303495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56399c8d-6089-42de-ba74-9b4104f1879b admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.304165 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56399c8d-6089-42de-ba74-9b4104f1879b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.309463 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56399c8d-6089-42de-ba74-9b4104f1879b admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.309671 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d31e5c9-584a-4c91-b0c7-a2187618250b admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.314467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134e307-5a04-4f04-a625-dd48a38bdc64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.315163 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134e307-5a04-4f04-a625-dd48a38bdc64 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.315588 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134e307-5a04-4f04-a625-dd48a38bdc64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.315877 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134e307-5a04-4f04-a625-dd48a38bdc64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.316607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134e307-5a04-4f04-a625-dd48a38bdc64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.318207 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134e307-5a04-4f04-a625-dd48a38bdc64 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.322779 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1293/2562] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:00.325473 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56399c8d-6089-42de-ba74-9b4104f1879b admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:00.334086 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1270/2563] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:08:00.336069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56399c8d-6089-42de-ba74-9b4104f1879b admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.337852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134e307-5a04-4f04-a625-dd48a38bdc64 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:00.339489 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-4134e307-5a04-4f04-a625-dd48a38bdc64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:00.341002 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a8483-7efb-4271-85fb-39b18a6afcbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.341667 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a8483-7efb-4271-85fb-39b18a6afcbf admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.341960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a8483-7efb-4271-85fb-39b18a6afcbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.342249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5a8483-7efb-4271-85fb-39b18a6afcbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.343010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a8483-7efb-4271-85fb-39b18a6afcbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.344614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5a8483-7efb-4271-85fb-39b18a6afcbf admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.351751 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134e307-5a04-4f04-a625-dd48a38bdc64 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.355798 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1294/2564] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:08:00.356684 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644d7ce-c64d-4fb8-8e91-5f3b691f7824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.357624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644d7ce-c64d-4fb8-8e91-5f3b691f7824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.357982 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644d7ce-c64d-4fb8-8e91-5f3b691f7824 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.358331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644d7ce-c64d-4fb8-8e91-5f3b691f7824 admin admin] PATH_INFO: `/v3/users/52c28aec93f8459e9d3216672c759b42` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.359492 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a8483-7efb-4271-85fb-39b18a6afcbf admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:00.360296 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644d7ce-c64d-4fb8-8e91-5f3b691f7824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52c28aec93f8459e9d3216672c759b42)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.362231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644d7ce-c64d-4fb8-8e91-5f3b691f7824 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.369646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5a8483-7efb-4271-85fb-39b18a6afcbf admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.374227 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39520f1d-ca56-40e7-aa12-7b3f48369fc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.375115 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39520f1d-ca56-40e7-aa12-7b3f48369fc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.375402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39520f1d-ca56-40e7-aa12-7b3f48369fc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.375676 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39520f1d-ca56-40e7-aa12-7b3f48369fc3 admin admin] PATH_INFO: `/v3/projects/51d1622a94da4621b94b0dcc82a5e64a/users/a1f73b00d23f46fbacfd571fe794f8ce/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.390943 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39520f1d-ca56-40e7-aa12-7b3f48369fc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51d1622a94da4621b94b0dcc82a5e64a, user_id=a1f73b00d23f46fbacfd571fe794f8ce, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.393595 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39520f1d-ca56-40e7-aa12-7b3f48369fc3 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.405658 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1295/2565] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:00 2022] PUT /identity/v3/projects/51d1622a94da4621b94b0dcc82a5e64a/users/a1f73b00d23f46fbacfd571fe794f8ce/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:00.406411 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39520f1d-ca56-40e7-aa12-7b3f48369fc3 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:00.417585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d644d7ce-c64d-4fb8-8e91-5f3b691f7824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52c28aec93f8459e9d3216672c759b42'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:00.418247 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d644d7ce-c64d-4fb8-8e91-5f3b691f7824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52c28aec93f8459e9d3216672c759b42'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:00.425581 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d644d7ce-c64d-4fb8-8e91-5f3b691f7824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52c28aec93f8459e9d3216672c759b42'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:00.426236 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d644d7ce-c64d-4fb8-8e91-5f3b691f7824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52c28aec93f8459e9d3216672c759b42'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:00.438963 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d644d7ce-c64d-4fb8-8e91-5f3b691f7824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52c28aec93f8459e9d3216672c759b42'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:00.452703 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1271/2566] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:00 2022] DELETE /identity/v3/users/52c28aec93f8459e9d3216672c759b42 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:00.454175 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed1603-7518-4fd5-bb8e-e48db54faa5e None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.454642 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed1603-7518-4fd5-bb8e-e48db54faa5e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.455102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ed1603-7518-4fd5-bb8e-e48db54faa5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.478226 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39520f1d-ca56-40e7-aa12-7b3f48369fc3 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.481967 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-e6ed1603-7518-4fd5-bb8e-e48db54faa5e None None] MFA Rules not processed for user `a1f73b00d23f46fbacfd571fe794f8ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:00.483310 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-e6ed1603-7518-4fd5-bb8e-e48db54faa5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:00.519558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:00.536718 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f0361-b33d-4ce8-a881-4eaa59426abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.537613 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f0361-b33d-4ce8-a881-4eaa59426abd admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.537905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f0361-b33d-4ce8-a881-4eaa59426abd admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.538190 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f0361-b33d-4ce8-a881-4eaa59426abd admin admin] PATH_INFO: `/v3/projects/106a057617484fd9b8f804c29365a485/users/0b46b1b9f9f9403ca8bbf6e0fc3a2a61/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.552906 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f0361-b33d-4ce8-a881-4eaa59426abd admin admin] RBAC: Authorizing `identity:create_grant(project_id=106a057617484fd9b8f804c29365a485, user_id=0b46b1b9f9f9403ca8bbf6e0fc3a2a61, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.553227 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1272/2567] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:00 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:00.554502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaa02c-1c32-4e9a-a4df-1dc81580ebbd None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.554820 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaa02c-1c32-4e9a-a4df-1dc81580ebbd None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.555101 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeaa02c-1c32-4e9a-a4df-1dc81580ebbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.557779 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f0361-b33d-4ce8-a881-4eaa59426abd admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.570953 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1296/2568] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:00 2022] PUT /identity/v3/projects/106a057617484fd9b8f804c29365a485/users/0b46b1b9f9f9403ca8bbf6e0fc3a2a61/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:00.571728 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f0361-b33d-4ce8-a881-4eaa59426abd admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:00.573096 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-ddeaa02c-1c32-4e9a-a4df-1dc81580ebbd None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:00.574287 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-ddeaa02c-1c32-4e9a-a4df-1dc81580ebbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:00.576133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-639f0361-b33d-4ce8-a881-4eaa59426abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:00.587707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:00.639375 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1273/2569] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:00.640662 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010adf7-72c5-4774-8877-8499b8f02943 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.640951 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010adf7-72c5-4774-8877-8499b8f02943 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.641225 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b010adf7-72c5-4774-8877-8499b8f02943 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.644471 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f0361-b33d-4ce8-a881-4eaa59426abd admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.649572 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e98516-e525-4637-8499-c35806bc7df5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.650245 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e98516-e525-4637-8499-c35806bc7df5 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.650527 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e98516-e525-4637-8499-c35806bc7df5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.650836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e98516-e525-4637-8499-c35806bc7df5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.651547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e98516-e525-4637-8499-c35806bc7df5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.653425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e98516-e525-4637-8499-c35806bc7df5 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.658271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-b010adf7-72c5-4774-8877-8499b8f02943 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:00.659482 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-b010adf7-72c5-4774-8877-8499b8f02943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:00.671579 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1274/2570] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:00.671579 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010adf7-72c5-4774-8877-8499b8f02943 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:00.671579 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-b010adf7-72c5-4774-8877-8499b8f02943 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:00.683775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1297/2571] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:00 2022] POST /identity/v3/projects => generated 401 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:00.683775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e98516-e525-4637-8499-c35806bc7df5 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:00.683775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-21e98516-e525-4637-8499-c35806bc7df5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:00.731709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b010adf7-72c5-4774-8877-8499b8f02943 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.735583 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a48228-6a20-419b-a898-25bf5e80b12c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.736239 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a48228-6a20-419b-a898-25bf5e80b12c service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.736521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a48228-6a20-419b-a898-25bf5e80b12c service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.736798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a48228-6a20-419b-a898-25bf5e80b12c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.738374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e98516-e525-4637-8499-c35806bc7df5 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.742010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d861a-188c-4ad6-80fa-a3749cc691fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.742719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d861a-188c-4ad6-80fa-a3749cc691fd service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.743510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d861a-188c-4ad6-80fa-a3749cc691fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.743790 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76d861a-188c-4ad6-80fa-a3749cc691fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.798771 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a48228-6a20-419b-a898-25bf5e80b12c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.800511 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90a48228-6a20-419b-a898-25bf5e80b12c service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.811857 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d861a-188c-4ad6-80fa-a3749cc691fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.812231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1275/2572] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:00.813310 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76d861a-188c-4ad6-80fa-a3749cc691fd service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.818567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:00.821785 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a48228-6a20-419b-a898-25bf5e80b12c service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:00.827995 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1298/2573] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:00.833190 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a48228-6a20-419b-a898-25bf5e80b12c service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.839067 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b5c4f-7fd9-484f-8636-c07f8b53c6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.839742 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5c4f-7fd9-484f-8636-c07f8b53c6b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.840030 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5c4f-7fd9-484f-8636-c07f8b53c6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.840438 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b5c4f-7fd9-484f-8636-c07f8b53c6b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.841134 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b5c4f-7fd9-484f-8636-c07f8b53c6b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.843161 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38b5c4f-7fd9-484f-8636-c07f8b53c6b6 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.874160 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1276/2574] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:00 2022] POST /identity/v3/users => generated 307 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:00.877075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d861a-188c-4ad6-80fa-a3749cc691fd service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:00.889391 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76d861a-188c-4ad6-80fa-a3749cc691fd service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.894975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933419f-5a90-411b-9177-49ed3fe2eee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.895988 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933419f-5a90-411b-9177-49ed3fe2eee5 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.896286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933419f-5a90-411b-9177-49ed3fe2eee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.897460 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e933419f-5a90-411b-9177-49ed3fe2eee5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.898213 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933419f-5a90-411b-9177-49ed3fe2eee5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.900967 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e933419f-5a90-411b-9177-49ed3fe2eee5 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.922059 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b5c4f-7fd9-484f-8636-c07f8b53c6b6 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:00.932951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1299/2575] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:00.936139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b5c4f-7fd9-484f-8636-c07f8b53c6b6 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:00.944813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5908956-fa2e-4a9e-9eff-5a38144a6cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:00.945913 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5908956-fa2e-4a9e-9eff-5a38144a6cd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:00.946182 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5908956-fa2e-4a9e-9eff-5a38144a6cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:00.946838 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5908956-fa2e-4a9e-9eff-5a38144a6cd4 admin admin] PATH_INFO: `/v3/projects/5d260517390f436495b4bc5e4e356728/users/797a9311ba0748299c7c9e6e6e3a1e98/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:00.972377 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5908956-fa2e-4a9e-9eff-5a38144a6cd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d260517390f436495b4bc5e4e356728, user_id=797a9311ba0748299c7c9e6e6e3a1e98, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:00.975825 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5908956-fa2e-4a9e-9eff-5a38144a6cd4 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:00.995831 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1277/2576] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:00 2022] PUT /identity/v3/projects/5d260517390f436495b4bc5e4e356728/users/797a9311ba0748299c7c9e6e6e3a1e98/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:00.998355 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933419f-5a90-411b-9177-49ed3fe2eee5 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:01.010473 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e933419f-5a90-411b-9177-49ed3fe2eee5 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:01.119020 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18308e5e-cc54-4016-965a-0e125b4e2ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:01.119761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18308e5e-cc54-4016-965a-0e125b4e2ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:01.120179 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18308e5e-cc54-4016-965a-0e125b4e2ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:01.120460 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18308e5e-cc54-4016-965a-0e125b4e2ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:01.121381 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18308e5e-cc54-4016-965a-0e125b4e2ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:01.123143 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18308e5e-cc54-4016-965a-0e125b4e2ac1 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:01.138532 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1300/2577] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:08:01.140119 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5908956-fa2e-4a9e-9eff-5a38144a6cd4 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:01.154105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5908956-fa2e-4a9e-9eff-5a38144a6cd4 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:01.161797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2d91c-1083-49ba-890a-6bcf1ac68da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:01.162966 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2d91c-1083-49ba-890a-6bcf1ac68da4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:01.163260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2d91c-1083-49ba-890a-6bcf1ac68da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:01.163546 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b2d91c-1083-49ba-890a-6bcf1ac68da4 admin admin] PATH_INFO: `/v3/projects/5d260517390f436495b4bc5e4e356728/users/797a9311ba0748299c7c9e6e6e3a1e98/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:01.168322 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2d91c-1083-49ba-890a-6bcf1ac68da4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d260517390f436495b4bc5e4e356728, user_id=797a9311ba0748299c7c9e6e6e3a1e98, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:01.172528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b2d91c-1083-49ba-890a-6bcf1ac68da4 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:01.191902 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1278/2578] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:01 2022] PUT /identity/v3/projects/5d260517390f436495b4bc5e4e356728/users/797a9311ba0748299c7c9e6e6e3a1e98/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:01.202972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26230518-a3c5-4d05-a917-5fb0aa3ab905 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:01.203310 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26230518-a3c5-4d05-a917-5fb0aa3ab905 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:01.203545 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26230518-a3c5-4d05-a917-5fb0aa3ab905 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:01.222724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2d91c-1083-49ba-890a-6bcf1ac68da4 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:01.228426 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-e7b2d91c-1083-49ba-890a-6bcf1ac68da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:01.244604 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-26230518-a3c5-4d05-a917-5fb0aa3ab905 None None] MFA Rules not processed for user `797a9311ba0748299c7c9e6e6e3a1e98`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:01.246425 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-26230518-a3c5-4d05-a917-5fb0aa3ab905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:01.248816 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:01.341734 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b2d91c-1083-49ba-890a-6bcf1ac68da4 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:01.348028 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5062e14-e43e-4dff-bdd3-50ee1e2bd6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:01.349036 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5062e14-e43e-4dff-bdd3-50ee1e2bd6e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:01.349258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5062e14-e43e-4dff-bdd3-50ee1e2bd6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:01.349542 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5062e14-e43e-4dff-bdd3-50ee1e2bd6e9 admin admin] PATH_INFO: `/v3/projects/c5dcade3e63a44e6a998e8949ccce206` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:01.351361 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5062e14-e43e-4dff-bdd3-50ee1e2bd6e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5dcade3e63a44e6a998e8949ccce206)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:01.353948 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5062e14-e43e-4dff-bdd3-50ee1e2bd6e9 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:01.364720 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1301/2579] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:01 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:01.365661 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26230518-a3c5-4d05-a917-5fb0aa3ab905 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:01.367480 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-26230518-a3c5-4d05-a917-5fb0aa3ab905 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:01.419523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c5062e14-e43e-4dff-bdd3-50ee1e2bd6e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5dcade3e63a44e6a998e8949ccce206', 'request_id': 'req-c5062e14-e43e-4dff-bdd3-50ee1e2bd6e9'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:01.427025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c5062e14-e43e-4dff-bdd3-50ee1e2bd6e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5dcade3e63a44e6a998e8949ccce206', 'request_id': 'req-c5062e14-e43e-4dff-bdd3-50ee1e2bd6e9'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:01.431726 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26230518-a3c5-4d05-a917-5fb0aa3ab905 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:01.434885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c5062e14-e43e-4dff-bdd3-50ee1e2bd6e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5dcade3e63a44e6a998e8949ccce206', 'request_id': 'req-c5062e14-e43e-4dff-bdd3-50ee1e2bd6e9'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:01.462569 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:01.465452 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c5062e14-e43e-4dff-bdd3-50ee1e2bd6e9 admin admin] The token cache is being invalidate because project c5dcade3e63a44e6a998e8949ccce206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:01.466041 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c5062e14-e43e-4dff-bdd3-50ee1e2bd6e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:01.466860 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c5062e14-e43e-4dff-bdd3-50ee1e2bd6e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:01.469360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1279/2580] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:01 2022] DELETE /identity/v3/projects/c5dcade3e63a44e6a998e8949ccce206 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:01.474533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5062e14-e43e-4dff-bdd3-50ee1e2bd6e9 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:01.476041 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c5062e14-e43e-4dff-bdd3-50ee1e2bd6e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:01.490602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dabfbe-2a32-48ba-b802-1d184473a31d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:01.491292 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dabfbe-2a32-48ba-b802-1d184473a31d service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:01.491661 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dabfbe-2a32-48ba-b802-1d184473a31d service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:01.491961 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dabfbe-2a32-48ba-b802-1d184473a31d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:01.494285 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-d4dabfbe-2a32-48ba-b802-1d184473a31d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:01.552620 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5062e14-e43e-4dff-bdd3-50ee1e2bd6e9 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:01.559903 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6fa86-cb13-4832-922b-47110cf12932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:01.560887 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6fa86-cb13-4832-922b-47110cf12932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:01.561291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6fa86-cb13-4832-922b-47110cf12932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:01.561583 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d6fa86-cb13-4832-922b-47110cf12932 admin admin] PATH_INFO: `/v3/users/3f86e68ff70d4067a8e2ea3a60d91336` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:01.563615 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6fa86-cb13-4832-922b-47110cf12932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f86e68ff70d4067a8e2ea3a60d91336)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:01.565619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d6fa86-cb13-4832-922b-47110cf12932 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:01.573146 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dabfbe-2a32-48ba-b802-1d184473a31d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:01.574610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dabfbe-2a32-48ba-b802-1d184473a31d service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:01.588432 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1302/2581] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:01 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:01.608841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921dc775-0217-4a4e-a056-8eda945fce2e None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:01.609184 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921dc775-0217-4a4e-a056-8eda945fce2e None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:01.609478 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921dc775-0217-4a4e-a056-8eda945fce2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:01.616669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:01.624994 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d4d6fa86-cb13-4832-922b-47110cf12932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f86e68ff70d4067a8e2ea3a60d91336'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:01.626909 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d4d6fa86-cb13-4832-922b-47110cf12932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f86e68ff70d4067a8e2ea3a60d91336'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:01.638909 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d4d6fa86-cb13-4832-922b-47110cf12932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f86e68ff70d4067a8e2ea3a60d91336'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:01.640047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d4d6fa86-cb13-4832-922b-47110cf12932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f86e68ff70d4067a8e2ea3a60d91336'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:01.650723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-921dc775-0217-4a4e-a056-8eda945fce2e None None] MFA Rules not processed for user `0b46b1b9f9f9403ca8bbf6e0fc3a2a61`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:01.652317 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-921dc775-0217-4a4e-a056-8eda945fce2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:01.660559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d4d6fa86-cb13-4832-922b-47110cf12932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f86e68ff70d4067a8e2ea3a60d91336'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:01.679389 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1280/2582] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:01 2022] DELETE /identity/v3/users/3f86e68ff70d4067a8e2ea3a60d91336 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:01.697428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6fa86-cb13-4832-922b-47110cf12932 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:01.698494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d4d6fa86-cb13-4832-922b-47110cf12932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:01.741085 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1303/2583] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:01 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:01.746662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921dc775-0217-4a4e-a056-8eda945fce2e None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:01.752778 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-921dc775-0217-4a4e-a056-8eda945fce2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:01.769023 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921dc775-0217-4a4e-a056-8eda945fce2e None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:01.779376 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d6fa86-cb13-4832-922b-47110cf12932 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:01.784782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a57f0-249d-4957-b2e7-027924eb0bbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:01.786091 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a57f0-249d-4957-b2e7-027924eb0bbe service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:01.786630 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a57f0-249d-4957-b2e7-027924eb0bbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:01.787123 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110a57f0-249d-4957-b2e7-027924eb0bbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:01.790464 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-110a57f0-249d-4957-b2e7-027924eb0bbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:01.848106 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92716cd3-c689-4b14-8f53-29f4f8282d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:01.848698 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92716cd3-c689-4b14-8f53-29f4f8282d5c admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:01.848985 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92716cd3-c689-4b14-8f53-29f4f8282d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:01.849265 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92716cd3-c689-4b14-8f53-29f4f8282d5c admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:01.850363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92716cd3-c689-4b14-8f53-29f4f8282d5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:01.853342 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92716cd3-c689-4b14-8f53-29f4f8282d5c admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:01.875194 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a57f0-249d-4957-b2e7-027924eb0bbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:01.876652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110a57f0-249d-4957-b2e7-027924eb0bbe service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:01.877237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:01.887043 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1281/2584] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:08:01.898286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1304/2585] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:01 2022] POST /identity/v3/projects => generated 403 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:08:01.903469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a57f0-249d-4957-b2e7-027924eb0bbe service cinder] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:01.915993 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110a57f0-249d-4957-b2e7-027924eb0bbe service cinder] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:01.979894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b892a-5bb4-4d57-852a-5b2a22c265e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:01.980562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b892a-5bb4-4d57-852a-5b2a22c265e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:01.981002 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b892a-5bb4-4d57-852a-5b2a22c265e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:01.981437 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556b892a-5bb4-4d57-852a-5b2a22c265e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:01.982147 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b892a-5bb4-4d57-852a-5b2a22c265e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:01.984147 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556b892a-5bb4-4d57-852a-5b2a22c265e3 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:02.003567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92716cd3-c689-4b14-8f53-29f4f8282d5c admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:02.005351 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-92716cd3-c689-4b14-8f53-29f4f8282d5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:02.010486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1282/2586] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:01 2022] POST /identity/v3/users => generated 308 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:08:02.014309 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b892a-5bb4-4d57-852a-5b2a22c265e3 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:02.019693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:02.027821 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556b892a-5bb4-4d57-852a-5b2a22c265e3 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:02.036836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c60a50-d7f7-475f-96f4-327472c090f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:02.037688 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c60a50-d7f7-475f-96f4-327472c090f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:02.038212 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c60a50-d7f7-475f-96f4-327472c090f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:02.038827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92716cd3-c689-4b14-8f53-29f4f8282d5c admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:02.039047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c60a50-d7f7-475f-96f4-327472c090f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:02.040725 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c60a50-d7f7-475f-96f4-327472c090f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:02.043114 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c60a50-d7f7-475f-96f4-327472c090f1 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:02.047575 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e51448f-d4c0-49ca-8dd9-7253ca72dc89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:02.048679 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51448f-d4c0-49ca-8dd9-7253ca72dc89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:02.049685 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51448f-d4c0-49ca-8dd9-7253ca72dc89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:02.049975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e51448f-d4c0-49ca-8dd9-7253ca72dc89 admin admin] PATH_INFO: `/v3/projects/8b3aeb2c19014b43bdcd045b1a3991db` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:02.051412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51448f-d4c0-49ca-8dd9-7253ca72dc89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b3aeb2c19014b43bdcd045b1a3991db)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:02.053599 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e51448f-d4c0-49ca-8dd9-7253ca72dc89 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:02.058363 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1283/2587] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:02.059779 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c60a50-d7f7-475f-96f4-327472c090f1 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:02.070272 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c60a50-d7f7-475f-96f4-327472c090f1 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:02.076842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9dd4b0-2ead-4f8a-bbb3-7a24ddbdd115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:02.077735 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9dd4b0-2ead-4f8a-bbb3-7a24ddbdd115 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:02.078023 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9dd4b0-2ead-4f8a-bbb3-7a24ddbdd115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:02.078302 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9dd4b0-2ead-4f8a-bbb3-7a24ddbdd115 admin admin] PATH_INFO: `/v3/projects/fb1199adbd594f9eaa201525dd9880da/users/c502ec3a82ef477ebf5536cf00b684e7/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:02.104231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9dd4b0-2ead-4f8a-bbb3-7a24ddbdd115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb1199adbd594f9eaa201525dd9880da, user_id=c502ec3a82ef477ebf5536cf00b684e7, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:02.107142 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9dd4b0-2ead-4f8a-bbb3-7a24ddbdd115 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:02.112818 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-4e51448f-d4c0-49ca-8dd9-7253ca72dc89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b3aeb2c19014b43bdcd045b1a3991db', 'request_id': 'req-4e51448f-d4c0-49ca-8dd9-7253ca72dc89'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:02.120119 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-4e51448f-d4c0-49ca-8dd9-7253ca72dc89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b3aeb2c19014b43bdcd045b1a3991db', 'request_id': 'req-4e51448f-d4c0-49ca-8dd9-7253ca72dc89'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:02.122905 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1284/2588] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:02 2022] PUT /identity/v3/projects/fb1199adbd594f9eaa201525dd9880da/users/c502ec3a82ef477ebf5536cf00b684e7/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:02.126987 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9dd4b0-2ead-4f8a-bbb3-7a24ddbdd115 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:02.129445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-4e51448f-d4c0-49ca-8dd9-7253ca72dc89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b3aeb2c19014b43bdcd045b1a3991db', 'request_id': 'req-4e51448f-d4c0-49ca-8dd9-7253ca72dc89'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:02.143594 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-4e51448f-d4c0-49ca-8dd9-7253ca72dc89 admin admin] The token cache is being invalidate because project 8b3aeb2c19014b43bdcd045b1a3991db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:02.144148 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-4e51448f-d4c0-49ca-8dd9-7253ca72dc89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:02.144871 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-4e51448f-d4c0-49ca-8dd9-7253ca72dc89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:02.146948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1305/2589] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:02 2022] DELETE /identity/v3/projects/8b3aeb2c19014b43bdcd045b1a3991db => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:02.152809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e919f99-5c14-48cb-95b3-fe059b0be8d5 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:02.153122 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e919f99-5c14-48cb-95b3-fe059b0be8d5 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:02.153369 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e919f99-5c14-48cb-95b3-fe059b0be8d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:02.172928 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-8e919f99-5c14-48cb-95b3-fe059b0be8d5 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:02.174121 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-8e919f99-5c14-48cb-95b3-fe059b0be8d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:02.232501 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9dd4b0-2ead-4f8a-bbb3-7a24ddbdd115 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:02.244049 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a06cf9-52dc-449e-ae3f-77ee532c405b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:02.244874 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a06cf9-52dc-449e-ae3f-77ee532c405b admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:02.245162 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a06cf9-52dc-449e-ae3f-77ee532c405b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:02.245442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a06cf9-52dc-449e-ae3f-77ee532c405b admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:02.246180 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a06cf9-52dc-449e-ae3f-77ee532c405b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:02.249371 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a06cf9-52dc-449e-ae3f-77ee532c405b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:02.252309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1306/2590] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:02.262252 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1285/2591] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:08:02.265819 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dfe64-c23e-4fff-9afb-cbcdbf9ea9b1 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:02.266127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dfe64-c23e-4fff-9afb-cbcdbf9ea9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:02.266411 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347dfe64-c23e-4fff-9afb-cbcdbf9ea9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:02.266523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a06cf9-52dc-449e-ae3f-77ee532c405b admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:02.268308 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-a0a06cf9-52dc-449e-ae3f-77ee532c405b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:02.290830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a06cf9-52dc-449e-ae3f-77ee532c405b admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:02.292859 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96634b-ac87-4ffb-8d08-a8c7f663d85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:02.293812 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96634b-ac87-4ffb-8d08-a8c7f663d85e admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:02.294099 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96634b-ac87-4ffb-8d08-a8c7f663d85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:02.294376 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f96634b-ac87-4ffb-8d08-a8c7f663d85e admin admin] PATH_INFO: `/v3/projects/fb1199adbd594f9eaa201525dd9880da/users/c502ec3a82ef477ebf5536cf00b684e7/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:02.296429 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96634b-ac87-4ffb-8d08-a8c7f663d85e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb1199adbd594f9eaa201525dd9880da, user_id=c502ec3a82ef477ebf5536cf00b684e7, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:02.298780 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-347dfe64-c23e-4fff-9afb-cbcdbf9ea9b1 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:02.299141 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f96634b-ac87-4ffb-8d08-a8c7f663d85e admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:02.299819 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-347dfe64-c23e-4fff-9afb-cbcdbf9ea9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:02.314662 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1286/2592] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:02 2022] PUT /identity/v3/projects/fb1199adbd594f9eaa201525dd9880da/users/c502ec3a82ef477ebf5536cf00b684e7/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:02.388711 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1307/2593] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:02.393402 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96634b-ac87-4ffb-8d08-a8c7f663d85e admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:02.404247 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f96634b-ac87-4ffb-8d08-a8c7f663d85e admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:02.409754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e88ae6-9f2d-478f-b643-d0d11d61c567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:02.410413 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e88ae6-9f2d-478f-b643-d0d11d61c567 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:02.410746 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e88ae6-9f2d-478f-b643-d0d11d61c567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:02.411032 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e88ae6-9f2d-478f-b643-d0d11d61c567 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:02.411753 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e88ae6-9f2d-478f-b643-d0d11d61c567 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:08:02.412061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e88ae6-9f2d-478f-b643-d0d11d61c567 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:02.413916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e88ae6-9f2d-478f-b643-d0d11d61c567 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:02.427990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1287/2594] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:08:02.433194 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347dfe64-c23e-4fff-9afb-cbcdbf9ea9b1 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:02.442353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347dfe64-c23e-4fff-9afb-cbcdbf9ea9b1 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:02.448173 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83657227-d7b3-4e3b-ad4f-8bed54cbd913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:02.448887 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83657227-d7b3-4e3b-ad4f-8bed54cbd913 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:02.449289 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83657227-d7b3-4e3b-ad4f-8bed54cbd913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:02.449648 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83657227-d7b3-4e3b-ad4f-8bed54cbd913 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:02.450507 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83657227-d7b3-4e3b-ad4f-8bed54cbd913 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:02.452472 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83657227-d7b3-4e3b-ad4f-8bed54cbd913 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:02.466470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:02.484351 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1308/2595] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:02 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:02.485777 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e88ae6-9f2d-478f-b643-d0d11d61c567 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:02.508880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e88ae6-9f2d-478f-b643-d0d11d61c567 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:02.570625 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef9a51-1124-42a0-8ea2-fa51d212a677 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:02.572223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef9a51-1124-42a0-8ea2-fa51d212a677 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:02.572520 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef9a51-1124-42a0-8ea2-fa51d212a677 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:02.572786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef9a51-1124-42a0-8ea2-fa51d212a677 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:02.573478 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef9a51-1124-42a0-8ea2-fa51d212a677 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:02.575741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef9a51-1124-42a0-8ea2-fa51d212a677 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:02.598072 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1288/2596] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:02 2022] POST /identity/v3/users => generated 294 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:02.602518 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83657227-d7b3-4e3b-ad4f-8bed54cbd913 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:02.613959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83657227-d7b3-4e3b-ad4f-8bed54cbd913 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:02.620261 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a84c8-eb95-487a-ba9e-164dfdd83ea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:02.621011 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a84c8-eb95-487a-ba9e-164dfdd83ea3 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:02.621327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a84c8-eb95-487a-ba9e-164dfdd83ea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:02.621659 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105a84c8-eb95-487a-ba9e-164dfdd83ea3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:02.622532 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a84c8-eb95-487a-ba9e-164dfdd83ea3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:02.624400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105a84c8-eb95-487a-ba9e-164dfdd83ea3 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:02.648396 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1309/2597] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:02.650781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef9a51-1124-42a0-8ea2-fa51d212a677 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:02.662661 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef9a51-1124-42a0-8ea2-fa51d212a677 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:02.668115 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48047118-2fa9-444f-9574-aabbac906e89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:02.668808 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48047118-2fa9-444f-9574-aabbac906e89 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:02.669097 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48047118-2fa9-444f-9574-aabbac906e89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:02.669380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48047118-2fa9-444f-9574-aabbac906e89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:02.670286 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48047118-2fa9-444f-9574-aabbac906e89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:02.671828 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48047118-2fa9-444f-9574-aabbac906e89 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:02.688727 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1289/2598] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:02.689934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a84c8-eb95-487a-ba9e-164dfdd83ea3 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:02.702316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105a84c8-eb95-487a-ba9e-164dfdd83ea3 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:02.707189 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51b3a7-74bc-4ca9-b9d0-9829f7b4c97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:02.708083 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51b3a7-74bc-4ca9-b9d0-9829f7b4c97d admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:02.708534 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51b3a7-74bc-4ca9-b9d0-9829f7b4c97d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:02.708825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc51b3a7-74bc-4ca9-b9d0-9829f7b4c97d admin admin] PATH_INFO: `/v3/projects/3f19bd43d6c44eeaa4d0c5c681cebeae/users/63ddf828047548949f85d80807e74b4d/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:02.729138 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51b3a7-74bc-4ca9-b9d0-9829f7b4c97d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f19bd43d6c44eeaa4d0c5c681cebeae, user_id=63ddf828047548949f85d80807e74b4d, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:02.732309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc51b3a7-74bc-4ca9-b9d0-9829f7b4c97d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:02.746851 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1310/2599] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:02 2022] PUT /identity/v3/projects/3f19bd43d6c44eeaa4d0c5c681cebeae/users/63ddf828047548949f85d80807e74b4d/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:02.757037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f77d00-a0ee-41c0-9afc-8e5454109a05 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:02.757382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f77d00-a0ee-41c0-9afc-8e5454109a05 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:02.757600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f77d00-a0ee-41c0-9afc-8e5454109a05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:02.784105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-46f77d00-a0ee-41c0-9afc-8e5454109a05 None None] MFA Rules not processed for user `63ddf828047548949f85d80807e74b4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:02.785316 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-46f77d00-a0ee-41c0-9afc-8e5454109a05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:02.789296 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:02.853893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1290/2600] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:02.858353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51b3a7-74bc-4ca9-b9d0-9829f7b4c97d admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:02.930152 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc51b3a7-74bc-4ca9-b9d0-9829f7b4c97d admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:02.936865 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf172eb-4383-4dfe-b800-db3fd87878d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:02.937754 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf172eb-4383-4dfe-b800-db3fd87878d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:02.938046 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf172eb-4383-4dfe-b800-db3fd87878d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:02.938339 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf172eb-4383-4dfe-b800-db3fd87878d9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:02.939201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf172eb-4383-4dfe-b800-db3fd87878d9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:02.941117 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf172eb-4383-4dfe-b800-db3fd87878d9 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:02.971882 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970fa52-60d0-4540-8dcc-6fe2e0e4535a None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:02.972180 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970fa52-60d0-4540-8dcc-6fe2e0e4535a None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:02.972453 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970fa52-60d0-4540-8dcc-6fe2e0e4535a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:02.978202 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1311/2601] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:02 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:02.979270 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf172eb-4383-4dfe-b800-db3fd87878d9 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:03.003980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-8970fa52-60d0-4540-8dcc-6fe2e0e4535a None None] MFA Rules not processed for user `c502ec3a82ef477ebf5536cf00b684e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:03.005623 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-8970fa52-60d0-4540-8dcc-6fe2e0e4535a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:03.053701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf172eb-4383-4dfe-b800-db3fd87878d9 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:03.061313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5cfb2-8c10-4ae7-9d2d-69a95b75f020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:03.062004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5cfb2-8c10-4ae7-9d2d-69a95b75f020 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:03.062371 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5cfb2-8c10-4ae7-9d2d-69a95b75f020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:03.062748 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e5cfb2-8c10-4ae7-9d2d-69a95b75f020 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:03.063507 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5cfb2-8c10-4ae7-9d2d-69a95b75f020 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:03.065791 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e5cfb2-8c10-4ae7-9d2d-69a95b75f020 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:03.088154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1291/2602] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:02 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:03.093310 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1312/2603] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:02 2022] POST /identity/v3/users => generated 293 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:03.097201 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970fa52-60d0-4540-8dcc-6fe2e0e4535a None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:03.107895 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970fa52-60d0-4540-8dcc-6fe2e0e4535a None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:03.112526 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f454e-42de-4a05-ba05-f8de328a6f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:03.113200 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f454e-42de-4a05-ba05-f8de328a6f58 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:03.113495 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f454e-42de-4a05-ba05-f8de328a6f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:03.113779 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8f454e-42de-4a05-ba05-f8de328a6f58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:03.114509 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f454e-42de-4a05-ba05-f8de328a6f58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:03.116159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8f454e-42de-4a05-ba05-f8de328a6f58 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:03.127731 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1292/2604] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:03.131931 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5cfb2-8c10-4ae7-9d2d-69a95b75f020 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:03.146027 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e5cfb2-8c10-4ae7-9d2d-69a95b75f020 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:03.151288 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e60bc8-9703-4307-96df-87dacd539049 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:03.151970 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e60bc8-9703-4307-96df-87dacd539049 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:03.152264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e60bc8-9703-4307-96df-87dacd539049 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:03.152552 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e60bc8-9703-4307-96df-87dacd539049 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:03.153280 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e60bc8-9703-4307-96df-87dacd539049 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:03.154931 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e60bc8-9703-4307-96df-87dacd539049 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:03.166952 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1313/2605] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:08:03.171019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f454e-42de-4a05-ba05-f8de328a6f58 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:03.181514 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8f454e-42de-4a05-ba05-f8de328a6f58 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:03.186578 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c01195-c954-4982-9439-cb61745c8fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:03.187494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c01195-c954-4982-9439-cb61745c8fb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:03.187780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c01195-c954-4982-9439-cb61745c8fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:03.188062 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c01195-c954-4982-9439-cb61745c8fb7 admin admin] PATH_INFO: `/v3/projects/e578f3aba1d743a3a6fe5e6ae8ab3548/users/6e62eaf20c45403791a0f80f34ee4f08/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:03.203821 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c01195-c954-4982-9439-cb61745c8fb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e578f3aba1d743a3a6fe5e6ae8ab3548, user_id=6e62eaf20c45403791a0f80f34ee4f08, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:03.206510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c01195-c954-4982-9439-cb61745c8fb7 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:03.220778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1293/2606] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:03 2022] PUT /identity/v3/projects/e578f3aba1d743a3a6fe5e6ae8ab3548/users/6e62eaf20c45403791a0f80f34ee4f08/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:03.235977 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b079b-9552-442a-8b4b-a02b80cc94ee None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:03.236248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b079b-9552-442a-8b4b-a02b80cc94ee None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:03.236536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897b079b-9552-442a-8b4b-a02b80cc94ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:03.265296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-897b079b-9552-442a-8b4b-a02b80cc94ee None None] MFA Rules not processed for user `6e62eaf20c45403791a0f80f34ee4f08`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:03.266727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-897b079b-9552-442a-8b4b-a02b80cc94ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:03.276952 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:03.342918 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1314/2607] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:03 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:03.354925 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c01195-c954-4982-9439-cb61745c8fb7 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:03.416462 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:03.424984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c01195-c954-4982-9439-cb61745c8fb7 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:03.433304 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713706a-67d5-4c4c-9469-ff977c45e808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:03.433998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713706a-67d5-4c4c-9469-ff977c45e808 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:03.434291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713706a-67d5-4c4c-9469-ff977c45e808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:03.434660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6713706a-67d5-4c4c-9469-ff977c45e808 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:03.435321 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713706a-67d5-4c4c-9469-ff977c45e808 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:03.437262 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6713706a-67d5-4c4c-9469-ff977c45e808 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:03.469997 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1294/2608] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:03 2022] POST /identity/v3/projects => generated 385 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:08:03.473941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b079b-9552-442a-8b4b-a02b80cc94ee None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:03.488612 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897b079b-9552-442a-8b4b-a02b80cc94ee None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:03.568450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2acc81-b736-4d6d-ab13-d082ba56eb75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:03.569289 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2acc81-b736-4d6d-ab13-d082ba56eb75 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:03.569587 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2acc81-b736-4d6d-ab13-d082ba56eb75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:03.569875 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2acc81-b736-4d6d-ab13-d082ba56eb75 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:03.570694 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2acc81-b736-4d6d-ab13-d082ba56eb75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:03.572917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2acc81-b736-4d6d-ab13-d082ba56eb75 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:03.600387 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1315/2609] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:03 2022] POST /identity/v3/users => generated 299 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:08:03.605917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713706a-67d5-4c4c-9469-ff977c45e808 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:03.619864 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6713706a-67d5-4c4c-9469-ff977c45e808 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:03.628777 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbc996-212e-4b96-9865-065a029c0e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:03.629886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbc996-212e-4b96-9865-065a029c0e6e admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:03.630188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbc996-212e-4b96-9865-065a029c0e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:03.630648 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbbc996-212e-4b96-9865-065a029c0e6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:03.631463 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbc996-212e-4b96-9865-065a029c0e6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:03.633638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbbc996-212e-4b96-9865-065a029c0e6e admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:03.653948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2acc81-b736-4d6d-ab13-d082ba56eb75 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:03.663521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1295/2610] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:03.669286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2acc81-b736-4d6d-ab13-d082ba56eb75 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:03.677932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132cbdc-180e-4f3a-92d9-a10b5d7d50cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:03.679077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132cbdc-180e-4f3a-92d9-a10b5d7d50cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:03.684502 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132cbdc-180e-4f3a-92d9-a10b5d7d50cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:03.684854 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0132cbdc-180e-4f3a-92d9-a10b5d7d50cc admin admin] PATH_INFO: `/v3/projects/122c5451dae941cca46a2c253b23a2d7/users/4f2913ad5b8f440a9ec71a507b43fc0f/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:03.707832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132cbdc-180e-4f3a-92d9-a10b5d7d50cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=122c5451dae941cca46a2c253b23a2d7, user_id=4f2913ad5b8f440a9ec71a507b43fc0f, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:03.711285 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0132cbdc-180e-4f3a-92d9-a10b5d7d50cc admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:03.733432 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1316/2611] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:03 2022] PUT /identity/v3/projects/122c5451dae941cca46a2c253b23a2d7/users/4f2913ad5b8f440a9ec71a507b43fc0f/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:03.746394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbc996-212e-4b96-9865-065a029c0e6e admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:03.764526 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbbc996-212e-4b96-9865-065a029c0e6e admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:03.835242 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0fd85-541a-46cd-98a5-5f511df625a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:03.836018 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0fd85-541a-46cd-98a5-5f511df625a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:03.836951 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0fd85-541a-46cd-98a5-5f511df625a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:03.837241 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0fd85-541a-46cd-98a5-5f511df625a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:03.838646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0fd85-541a-46cd-98a5-5f511df625a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:03.840876 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c0fd85-541a-46cd-98a5-5f511df625a6 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:03.851591 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1296/2612] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:03.855655 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132cbdc-180e-4f3a-92d9-a10b5d7d50cc admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:03.866546 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0132cbdc-180e-4f3a-92d9-a10b5d7d50cc admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:03.872198 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8108b7-4060-4a3e-bd52-eec516dbdc2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:03.873182 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8108b7-4060-4a3e-bd52-eec516dbdc2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:03.873466 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8108b7-4060-4a3e-bd52-eec516dbdc2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:03.873745 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8108b7-4060-4a3e-bd52-eec516dbdc2e admin admin] PATH_INFO: `/v3/projects/122c5451dae941cca46a2c253b23a2d7/users/4f2913ad5b8f440a9ec71a507b43fc0f/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:03.875829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8108b7-4060-4a3e-bd52-eec516dbdc2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=122c5451dae941cca46a2c253b23a2d7, user_id=4f2913ad5b8f440a9ec71a507b43fc0f, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:03.878484 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8108b7-4060-4a3e-bd52-eec516dbdc2e admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:03.894756 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1317/2613] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:03 2022] PUT /identity/v3/projects/122c5451dae941cca46a2c253b23a2d7/users/4f2913ad5b8f440a9ec71a507b43fc0f/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:03.905403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a45c1d-23df-4768-85f7-49bdf604abe8 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:03.905708 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a45c1d-23df-4768-85f7-49bdf604abe8 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:03.905980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a45c1d-23df-4768-85f7-49bdf604abe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:03.937454 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-e4a45c1d-23df-4768-85f7-49bdf604abe8 None None] MFA Rules not processed for user `4f2913ad5b8f440a9ec71a507b43fc0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:03.939185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-e4a45c1d-23df-4768-85f7-49bdf604abe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:04.019099 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1297/2614] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:03 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:04.030017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8108b7-4060-4a3e-bd52-eec516dbdc2e admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:04.031861 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ff8108b7-4060-4a3e-bd52-eec516dbdc2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:04.089706 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8108b7-4060-4a3e-bd52-eec516dbdc2e admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:04.093644 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c22dfa-be85-4403-ac66-1e6d02d95c2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:04.094308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c22dfa-be85-4403-ac66-1e6d02d95c2f service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:04.094618 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c22dfa-be85-4403-ac66-1e6d02d95c2f service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:04.094905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c22dfa-be85-4403-ac66-1e6d02d95c2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:04.104699 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c22dfa-be85-4403-ac66-1e6d02d95c2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:04.106137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c22dfa-be85-4403-ac66-1e6d02d95c2f service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:04.117895 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1318/2615] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:04 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:04.337203 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a45c1d-23df-4768-85f7-49bdf604abe8 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:04.398042 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:04.410263 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a45c1d-23df-4768-85f7-49bdf604abe8 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:04.416564 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1fdae-d540-4d61-bca4-cf95ae6956bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:04.417517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1fdae-d540-4d61-bca4-cf95ae6956bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:04.417835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1fdae-d540-4d61-bca4-cf95ae6956bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:04.418135 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc1fdae-d540-4d61-bca4-cf95ae6956bb admin admin] PATH_INFO: `/v3/users/0b46b1b9f9f9403ca8bbf6e0fc3a2a61` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:04.419540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1fdae-d540-4d61-bca4-cf95ae6956bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b46b1b9f9f9403ca8bbf6e0fc3a2a61)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:04.421507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc1fdae-d540-4d61-bca4-cf95ae6956bb admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:04.475327 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2dc1fdae-d540-4d61-bca4-cf95ae6956bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b46b1b9f9f9403ca8bbf6e0fc3a2a61'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:04.476253 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2dc1fdae-d540-4d61-bca4-cf95ae6956bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b46b1b9f9f9403ca8bbf6e0fc3a2a61'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:04.484885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2dc1fdae-d540-4d61-bca4-cf95ae6956bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b46b1b9f9f9403ca8bbf6e0fc3a2a61'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:04.485563 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2dc1fdae-d540-4d61-bca4-cf95ae6956bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b46b1b9f9f9403ca8bbf6e0fc3a2a61'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:04.498561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2dc1fdae-d540-4d61-bca4-cf95ae6956bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b46b1b9f9f9403ca8bbf6e0fc3a2a61'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:04.514213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1298/2616] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:04 2022] DELETE /identity/v3/users/0b46b1b9f9f9403ca8bbf6e0fc3a2a61 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:04.694412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c22dfa-be85-4403-ac66-1e6d02d95c2f service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:04.696982 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c1c22dfa-be85-4403-ac66-1e6d02d95c2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:04.703784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1fdae-d540-4d61-bca4-cf95ae6956bb admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:04.705206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-2dc1fdae-d540-4d61-bca4-cf95ae6956bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:04.710864 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c22dfa-be85-4403-ac66-1e6d02d95c2f service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:04.717912 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:04.784839 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc1fdae-d540-4d61-bca4-cf95ae6956bb admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:04.785747 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06060b14-cb30-45d0-a5d9-217f16e906f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:04.786563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06060b14-cb30-45d0-a5d9-217f16e906f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:04.786883 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06060b14-cb30-45d0-a5d9-217f16e906f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:04.787192 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06060b14-cb30-45d0-a5d9-217f16e906f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:04.789134 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-06060b14-cb30-45d0-a5d9-217f16e906f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:04.789684 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593c62c-dd2d-442d-aa52-4c6fe3294021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:04.790541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593c62c-dd2d-442d-aa52-4c6fe3294021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:04.790927 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593c62c-dd2d-442d-aa52-4c6fe3294021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:04.791192 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9593c62c-dd2d-442d-aa52-4c6fe3294021 admin admin] PATH_INFO: `/v3/projects/106a057617484fd9b8f804c29365a485` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:04.792366 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593c62c-dd2d-442d-aa52-4c6fe3294021 admin admin] RBAC: Authorizing `identity:delete_project(project_id=106a057617484fd9b8f804c29365a485)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:04.794382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9593c62c-dd2d-442d-aa52-4c6fe3294021 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:04.873088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06060b14-cb30-45d0-a5d9-217f16e906f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:04.873215 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-9593c62c-dd2d-442d-aa52-4c6fe3294021 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '106a057617484fd9b8f804c29365a485', 'request_id': 'req-9593c62c-dd2d-442d-aa52-4c6fe3294021'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:04.873308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06060b14-cb30-45d0-a5d9-217f16e906f3 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:04.873400 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-9593c62c-dd2d-442d-aa52-4c6fe3294021 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '106a057617484fd9b8f804c29365a485', 'request_id': 'req-9593c62c-dd2d-442d-aa52-4c6fe3294021'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:04.876526 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1319/2617] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:04.879804 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-9593c62c-dd2d-442d-aa52-4c6fe3294021 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '106a057617484fd9b8f804c29365a485', 'request_id': 'req-9593c62c-dd2d-442d-aa52-4c6fe3294021'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:04.896927 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:04.902154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-9593c62c-dd2d-442d-aa52-4c6fe3294021 admin admin] The token cache is being invalidate because project 106a057617484fd9b8f804c29365a485 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:04.902804 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-9593c62c-dd2d-442d-aa52-4c6fe3294021 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:04.903952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-9593c62c-dd2d-442d-aa52-4c6fe3294021 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:04.906105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1299/2618] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:04 2022] DELETE /identity/v3/projects/106a057617484fd9b8f804c29365a485 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:04.909924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06060b14-cb30-45d0-a5d9-217f16e906f3 service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:04.927262 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06060b14-cb30-45d0-a5d9-217f16e906f3 service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:05.008639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe4c0b3-1f20-4f72-b3b4-530ab7657598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:05.009629 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4c0b3-1f20-4f72-b3b4-530ab7657598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:05.009979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4c0b3-1f20-4f72-b3b4-530ab7657598 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:05.010457 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe4c0b3-1f20-4f72-b3b4-530ab7657598 admin admin] PATH_INFO: `/v3/users/c502ec3a82ef477ebf5536cf00b684e7` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:05.012078 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe4c0b3-1f20-4f72-b3b4-530ab7657598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c502ec3a82ef477ebf5536cf00b684e7)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:05.014145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe4c0b3-1f20-4f72-b3b4-530ab7657598 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:05.080445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-fbe4c0b3-1f20-4f72-b3b4-530ab7657598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c502ec3a82ef477ebf5536cf00b684e7'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:05.089932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-fbe4c0b3-1f20-4f72-b3b4-530ab7657598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c502ec3a82ef477ebf5536cf00b684e7'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:05.090753 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-fbe4c0b3-1f20-4f72-b3b4-530ab7657598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c502ec3a82ef477ebf5536cf00b684e7'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:05.097174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:05.106085 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-fbe4c0b3-1f20-4f72-b3b4-530ab7657598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c502ec3a82ef477ebf5536cf00b684e7'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:05.107309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-fbe4c0b3-1f20-4f72-b3b4-530ab7657598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c502ec3a82ef477ebf5536cf00b684e7'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:05.120825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1320/2619] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:04 2022] DELETE /identity/v3/users/c502ec3a82ef477ebf5536cf00b684e7 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:05.335694 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593c62c-dd2d-442d-aa52-4c6fe3294021 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:05.337766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-9593c62c-dd2d-442d-aa52-4c6fe3294021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:05.357003 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9593c62c-dd2d-442d-aa52-4c6fe3294021 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:05.412217 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bacf5f7-ca5c-41c5-84a3-f81722b788bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:05.413014 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacf5f7-ca5c-41c5-84a3-f81722b788bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:05.413297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacf5f7-ca5c-41c5-84a3-f81722b788bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:05.413573 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bacf5f7-ca5c-41c5-84a3-f81722b788bf admin admin] PATH_INFO: `/v3/projects/fb1199adbd594f9eaa201525dd9880da` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:05.414802 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacf5f7-ca5c-41c5-84a3-f81722b788bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb1199adbd594f9eaa201525dd9880da)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:05.416694 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bacf5f7-ca5c-41c5-84a3-f81722b788bf admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:05.463759 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5bacf5f7-ca5c-41c5-84a3-f81722b788bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb1199adbd594f9eaa201525dd9880da', 'request_id': 'req-5bacf5f7-ca5c-41c5-84a3-f81722b788bf'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:05.469802 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5bacf5f7-ca5c-41c5-84a3-f81722b788bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb1199adbd594f9eaa201525dd9880da', 'request_id': 'req-5bacf5f7-ca5c-41c5-84a3-f81722b788bf'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:05.475196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5bacf5f7-ca5c-41c5-84a3-f81722b788bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb1199adbd594f9eaa201525dd9880da', 'request_id': 'req-5bacf5f7-ca5c-41c5-84a3-f81722b788bf'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:05.488672 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5bacf5f7-ca5c-41c5-84a3-f81722b788bf admin admin] The token cache is being invalidate because project fb1199adbd594f9eaa201525dd9880da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:05.488962 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5bacf5f7-ca5c-41c5-84a3-f81722b788bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:05.489590 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5bacf5f7-ca5c-41c5-84a3-f81722b788bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:05.491240 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1300/2620] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:05 2022] DELETE /identity/v3/projects/fb1199adbd594f9eaa201525dd9880da => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:05.496576 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d2c70-5536-4ba8-8f7a-2509f7d05d67 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:05.496893 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d2c70-5536-4ba8-8f7a-2509f7d05d67 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:05.497178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16d2c70-5536-4ba8-8f7a-2509f7d05d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:05.514948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-f16d2c70-5536-4ba8-8f7a-2509f7d05d67 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:05.516157 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-f16d2c70-5536-4ba8-8f7a-2509f7d05d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:05.577668 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1321/2621] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:05.586496 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cc55a-3d69-48f2-908a-823f87147b9e None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:05.586812 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cc55a-3d69-48f2-908a-823f87147b9e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:05.587116 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177cc55a-3d69-48f2-908a-823f87147b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:05.605191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-177cc55a-3d69-48f2-908a-823f87147b9e None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:05.606395 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-177cc55a-3d69-48f2-908a-823f87147b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:05.615284 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1301/2622] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:05.620020 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d2c70-5536-4ba8-8f7a-2509f7d05d67 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:05.629848 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16d2c70-5536-4ba8-8f7a-2509f7d05d67 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:05.635040 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69956cb5-64f4-49a3-b6f1-c9da23ad7c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:05.635815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69956cb5-64f4-49a3-b6f1-c9da23ad7c84 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:05.636317 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69956cb5-64f4-49a3-b6f1-c9da23ad7c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:05.636813 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69956cb5-64f4-49a3-b6f1-c9da23ad7c84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:05.637922 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69956cb5-64f4-49a3-b6f1-c9da23ad7c84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:08:05.638476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69956cb5-64f4-49a3-b6f1-c9da23ad7c84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:05.640866 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69956cb5-64f4-49a3-b6f1-c9da23ad7c84 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:05.654394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1322/2623] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:08:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:08:05.658980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177cc55a-3d69-48f2-908a-823f87147b9e None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:05.668584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177cc55a-3d69-48f2-908a-823f87147b9e None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:05.673185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947830da-3e36-430f-9a18-c96cfce4e52b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:05.673837 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947830da-3e36-430f-9a18-c96cfce4e52b admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:05.674127 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947830da-3e36-430f-9a18-c96cfce4e52b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:05.674408 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947830da-3e36-430f-9a18-c96cfce4e52b admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:05.675134 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947830da-3e36-430f-9a18-c96cfce4e52b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:05.677028 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947830da-3e36-430f-9a18-c96cfce4e52b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:05.704430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1302/2624] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:05 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:05.708535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69956cb5-64f4-49a3-b6f1-c9da23ad7c84 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:05.718570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69956cb5-64f4-49a3-b6f1-c9da23ad7c84 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:05.777215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c3205-039a-40bb-99ce-b3e8036962ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:05.777889 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c3205-039a-40bb-99ce-b3e8036962ea admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:05.778175 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c3205-039a-40bb-99ce-b3e8036962ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:05.778454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c3205-039a-40bb-99ce-b3e8036962ea admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:05.779183 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c3205-039a-40bb-99ce-b3e8036962ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:05.781069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0c3205-039a-40bb-99ce-b3e8036962ea admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:05.805578 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1323/2625] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:05 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:05.809775 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947830da-3e36-430f-9a18-c96cfce4e52b admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:05.820992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947830da-3e36-430f-9a18-c96cfce4e52b admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:05.826085 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a159306-c3c7-4148-9d1b-a00c9a1b292f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:05.826786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a159306-c3c7-4148-9d1b-a00c9a1b292f admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:05.827079 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a159306-c3c7-4148-9d1b-a00c9a1b292f admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:05.827364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a159306-c3c7-4148-9d1b-a00c9a1b292f admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:05.828092 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a159306-c3c7-4148-9d1b-a00c9a1b292f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:05.829679 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a159306-c3c7-4148-9d1b-a00c9a1b292f admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:05.841368 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1303/2626] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:05.845333 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c3205-039a-40bb-99ce-b3e8036962ea admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:05.855822 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c3205-039a-40bb-99ce-b3e8036962ea admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:05.860482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae390b-4fe8-466b-b1da-8b9468bae63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:05.861137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae390b-4fe8-466b-b1da-8b9468bae63a admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:05.861423 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae390b-4fe8-466b-b1da-8b9468bae63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:05.861702 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae390b-4fe8-466b-b1da-8b9468bae63a admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:05.862421 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae390b-4fe8-466b-b1da-8b9468bae63a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:05.864032 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae390b-4fe8-466b-b1da-8b9468bae63a admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:05.874429 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1324/2627] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:05.878477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a159306-c3c7-4148-9d1b-a00c9a1b292f admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:05.889171 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a159306-c3c7-4148-9d1b-a00c9a1b292f admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:05.893801 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fe401-c65e-48cf-b055-9bedf1059387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:05.894711 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fe401-c65e-48cf-b055-9bedf1059387 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:05.895001 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fe401-c65e-48cf-b055-9bedf1059387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:05.895279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975fe401-c65e-48cf-b055-9bedf1059387 admin admin] PATH_INFO: `/v3/projects/fb1e8a75367d44eb9df63c5b20c50454/users/b64dc90488534a8d9e0e2e253456156b/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:05.911360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975fe401-c65e-48cf-b055-9bedf1059387 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb1e8a75367d44eb9df63c5b20c50454, user_id=b64dc90488534a8d9e0e2e253456156b, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:05.913980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975fe401-c65e-48cf-b055-9bedf1059387 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:05.928166 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1304/2628] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:05 2022] PUT /identity/v3/projects/fb1e8a75367d44eb9df63c5b20c50454/users/b64dc90488534a8d9e0e2e253456156b/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:05.938409 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd70b6-14f8-4b5d-b5e3-a54b576c7730 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:05.938746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd70b6-14f8-4b5d-b5e3-a54b576c7730 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:05.939039 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd70b6-14f8-4b5d-b5e3-a54b576c7730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:05.968093 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-50cd70b6-14f8-4b5d-b5e3-a54b576c7730 None None] MFA Rules not processed for user `b64dc90488534a8d9e0e2e253456156b`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:05.969338 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-50cd70b6-14f8-4b5d-b5e3-a54b576c7730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:06.005732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:06.041960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1325/2629] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:05 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:06.046941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fe401-c65e-48cf-b055-9bedf1059387 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:06.076649 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:06.114499 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975fe401-c65e-48cf-b055-9bedf1059387 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:06.120039 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3f9f9-8d2a-414d-a53a-df7eca3c21e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:06.120743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3f9f9-8d2a-414d-a53a-df7eca3c21e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:06.121032 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3f9f9-8d2a-414d-a53a-df7eca3c21e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:06.121314 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf3f9f9-8d2a-414d-a53a-df7eca3c21e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:06.122020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3f9f9-8d2a-414d-a53a-df7eca3c21e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:06.123955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf3f9f9-8d2a-414d-a53a-df7eca3c21e7 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:06.153145 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1305/2630] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:06 2022] POST /identity/v3/projects => generated 373 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:06.156843 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd70b6-14f8-4b5d-b5e3-a54b576c7730 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:06.175886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd70b6-14f8-4b5d-b5e3-a54b576c7730 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:06.248804 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c50d4-d6ce-4f07-a472-11fe725629f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:06.249476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c50d4-d6ce-4f07-a472-11fe725629f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:06.249764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c50d4-d6ce-4f07-a472-11fe725629f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:06.250046 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000c50d4-d6ce-4f07-a472-11fe725629f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:06.250781 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c50d4-d6ce-4f07-a472-11fe725629f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:06.252673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000c50d4-d6ce-4f07-a472-11fe725629f3 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:06.278968 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1326/2631] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:06 2022] POST /identity/v3/users => generated 293 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:06.282850 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3f9f9-8d2a-414d-a53a-df7eca3c21e7 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:06.295163 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf3f9f9-8d2a-414d-a53a-df7eca3c21e7 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:06.296645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c50d4-d6ce-4f07-a472-11fe725629f3 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:06.298327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-000c50d4-d6ce-4f07-a472-11fe725629f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:06.299433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337f729-2234-44ad-8237-70a397fd2c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:06.300264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337f729-2234-44ad-8237-70a397fd2c2c admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:06.300667 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337f729-2234-44ad-8237-70a397fd2c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:06.301061 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7337f729-2234-44ad-8237-70a397fd2c2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:06.302800 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337f729-2234-44ad-8237-70a397fd2c2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:06.303793 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7337f729-2234-44ad-8237-70a397fd2c2c admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:06.316075 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1306/2632] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:06.320019 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337f729-2234-44ad-8237-70a397fd2c2c admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:06.323699 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000c50d4-d6ce-4f07-a472-11fe725629f3 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:06.329646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efcd8a31-6538-4b50-880f-026943d67714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:06.330808 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd8a31-6538-4b50-880f-026943d67714 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:06.331129 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd8a31-6538-4b50-880f-026943d67714 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:06.331412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcd8a31-6538-4b50-880f-026943d67714 admin admin] PATH_INFO: `/v3/users/a1f73b00d23f46fbacfd571fe794f8ce` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:06.331676 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7337f729-2234-44ad-8237-70a397fd2c2c admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:06.332663 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd8a31-6538-4b50-880f-026943d67714 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1f73b00d23f46fbacfd571fe794f8ce)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:06.334689 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efcd8a31-6538-4b50-880f-026943d67714 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:06.336958 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc05fe-a8d4-4408-8aad-9d4c50a537a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:06.337963 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc05fe-a8d4-4408-8aad-9d4c50a537a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:06.338507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc05fe-a8d4-4408-8aad-9d4c50a537a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:06.338507 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc05fe-a8d4-4408-8aad-9d4c50a537a2 admin admin] PATH_INFO: `/v3/projects/b1b665b659c947f79983ee60a84dffae/users/89c41da62fc04f5094f8a15ee9d7ba5a/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:06.356387 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc05fe-a8d4-4408-8aad-9d4c50a537a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b665b659c947f79983ee60a84dffae, user_id=89c41da62fc04f5094f8a15ee9d7ba5a, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:06.359113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24cc05fe-a8d4-4408-8aad-9d4c50a537a2 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:06.376133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1307/2633] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:06 2022] PUT /identity/v3/projects/b1b665b659c947f79983ee60a84dffae/users/89c41da62fc04f5094f8a15ee9d7ba5a/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:06.380165 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc05fe-a8d4-4408-8aad-9d4c50a537a2 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:06.399383 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-efcd8a31-6538-4b50-880f-026943d67714 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1f73b00d23f46fbacfd571fe794f8ce'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:06.406584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-efcd8a31-6538-4b50-880f-026943d67714 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1f73b00d23f46fbacfd571fe794f8ce'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:06.407313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-efcd8a31-6538-4b50-880f-026943d67714 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1f73b00d23f46fbacfd571fe794f8ce'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:06.422125 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-efcd8a31-6538-4b50-880f-026943d67714 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1f73b00d23f46fbacfd571fe794f8ce'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:06.423088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-efcd8a31-6538-4b50-880f-026943d67714 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1f73b00d23f46fbacfd571fe794f8ce'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:06.446755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1327/2634] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:06 2022] DELETE /identity/v3/users/a1f73b00d23f46fbacfd571fe794f8ce => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:06.449201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3c86f-5f94-475d-a073-203bc5efc61b None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:06.454718 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3c86f-5f94-475d-a073-203bc5efc61b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:06.455148 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab3c86f-5f94-475d-a073-203bc5efc61b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:06.464912 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc05fe-a8d4-4408-8aad-9d4c50a537a2 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:06.478004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-1ab3c86f-5f94-475d-a073-203bc5efc61b None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:06.480750 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-1ab3c86f-5f94-475d-a073-203bc5efc61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:06.520932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:06.534635 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be6a75a-6cb8-4090-b03c-92d113ec2d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:06.535701 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6a75a-6cb8-4090-b03c-92d113ec2d00 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:06.535992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6a75a-6cb8-4090-b03c-92d113ec2d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:06.536274 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6a75a-6cb8-4090-b03c-92d113ec2d00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:06.537017 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be6a75a-6cb8-4090-b03c-92d113ec2d00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:06.538634 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be6a75a-6cb8-4090-b03c-92d113ec2d00 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:06.548669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1308/2635] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:06 2022] GET /identity/v3/roles => generated 1420 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:08:06.552052 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1328/2636] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:06.555117 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be6a75a-6cb8-4090-b03c-92d113ec2d00 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:06.555117 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-7be6a75a-6cb8-4090-b03c-92d113ec2d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:06.561467 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3c86f-5f94-475d-a073-203bc5efc61b None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:06.562829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-1ab3c86f-5f94-475d-a073-203bc5efc61b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:06.569627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be6a75a-6cb8-4090-b03c-92d113ec2d00 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:06.576971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b538db19-3721-4763-abda-0c59e8f56285 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:06.577957 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538db19-3721-4763-abda-0c59e8f56285 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:06.578258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538db19-3721-4763-abda-0c59e8f56285 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:06.578527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538db19-3721-4763-abda-0c59e8f56285 admin admin] PATH_INFO: `/v3/projects/b1b665b659c947f79983ee60a84dffae/users/89c41da62fc04f5094f8a15ee9d7ba5a/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:06.580813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538db19-3721-4763-abda-0c59e8f56285 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1b665b659c947f79983ee60a84dffae, user_id=89c41da62fc04f5094f8a15ee9d7ba5a, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:06.584000 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b538db19-3721-4763-abda-0c59e8f56285 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:06.596611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1309/2637] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:06 2022] PUT /identity/v3/projects/b1b665b659c947f79983ee60a84dffae/users/89c41da62fc04f5094f8a15ee9d7ba5a/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:06.606783 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d64541-d95a-437c-acd4-2b155ef711ec None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:06.607056 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d64541-d95a-437c-acd4-2b155ef711ec None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:06.607339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d64541-d95a-437c-acd4-2b155ef711ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:06.620473 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:06.622057 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab3c86f-5f94-475d-a073-203bc5efc61b None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:06.636633 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-05d64541-d95a-437c-acd4-2b155ef711ec None None] MFA Rules not processed for user `89c41da62fc04f5094f8a15ee9d7ba5a`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:06.638224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-05d64541-d95a-437c-acd4-2b155ef711ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:06.678620 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b3478-7e0b-4a1f-a058-e81ec635300b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:06.679320 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b3478-7e0b-4a1f-a058-e81ec635300b service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:06.680815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b3478-7e0b-4a1f-a058-e81ec635300b service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:06.681120 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b3478-7e0b-4a1f-a058-e81ec635300b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:06.716498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1310/2638] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:06.728710 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d64541-d95a-437c-acd4-2b155ef711ec None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:06.730216 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-05d64541-d95a-437c-acd4-2b155ef711ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:06.750997 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b3478-7e0b-4a1f-a058-e81ec635300b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:06.752464 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692b3478-7e0b-4a1f-a058-e81ec635300b service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:06.762627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1329/2639] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:08:06.790899 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d64541-d95a-437c-acd4-2b155ef711ec None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:06.794625 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb568777-d2de-430a-82e9-d42ff3f17f5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:06.795285 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb568777-d2de-430a-82e9-d42ff3f17f5e service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:06.795576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb568777-d2de-430a-82e9-d42ff3f17f5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:06.795854 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb568777-d2de-430a-82e9-d42ff3f17f5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:06.805251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb568777-d2de-430a-82e9-d42ff3f17f5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:06.806715 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb568777-d2de-430a-82e9-d42ff3f17f5e service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:06.816708 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1311/2640] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:06 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:08:07.072468 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b3478-7e0b-4a1f-a058-e81ec635300b service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:07.074336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-692b3478-7e0b-4a1f-a058-e81ec635300b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:07.109978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692b3478-7e0b-4a1f-a058-e81ec635300b service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:07.115008 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8fa1ef-5e71-4113-9a80-f0e7685e6d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:07.115942 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fa1ef-5e71-4113-9a80-f0e7685e6d5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:07.116231 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fa1ef-5e71-4113-9a80-f0e7685e6d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:07.116543 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8fa1ef-5e71-4113-9a80-f0e7685e6d5c admin admin] PATH_INFO: `/v3/projects/51d1622a94da4621b94b0dcc82a5e64a` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:07.118396 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8fa1ef-5e71-4113-9a80-f0e7685e6d5c admin admin] RBAC: Authorizing `identity:delete_project(project_id=51d1622a94da4621b94b0dcc82a5e64a)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:07.122419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8fa1ef-5e71-4113-9a80-f0e7685e6d5c admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:07.171558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:07.180523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-2c8fa1ef-5e71-4113-9a80-f0e7685e6d5c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51d1622a94da4621b94b0dcc82a5e64a', 'request_id': 'req-2c8fa1ef-5e71-4113-9a80-f0e7685e6d5c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.187515 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-2c8fa1ef-5e71-4113-9a80-f0e7685e6d5c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51d1622a94da4621b94b0dcc82a5e64a', 'request_id': 'req-2c8fa1ef-5e71-4113-9a80-f0e7685e6d5c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.193181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-2c8fa1ef-5e71-4113-9a80-f0e7685e6d5c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51d1622a94da4621b94b0dcc82a5e64a', 'request_id': 'req-2c8fa1ef-5e71-4113-9a80-f0e7685e6d5c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.210476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-2c8fa1ef-5e71-4113-9a80-f0e7685e6d5c admin admin] The token cache is being invalidate because project 51d1622a94da4621b94b0dcc82a5e64a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:07.211382 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-2c8fa1ef-5e71-4113-9a80-f0e7685e6d5c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.212056 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-2c8fa1ef-5e71-4113-9a80-f0e7685e6d5c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.213810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1330/2641] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:07 2022] DELETE /identity/v3/projects/51d1622a94da4621b94b0dcc82a5e64a => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:07.217662 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb568777-d2de-430a-82e9-d42ff3f17f5e service cinder] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:07.229913 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb568777-d2de-430a-82e9-d42ff3f17f5e service cinder] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:07.258861 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8fa1ef-5e71-4113-9a80-f0e7685e6d5c admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:07.260305 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-2c8fa1ef-5e71-4113-9a80-f0e7685e6d5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:07.291880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a738473f-42b0-4617-9047-7056a320e0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:07.292782 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738473f-42b0-4617-9047-7056a320e0e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:07.293477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738473f-42b0-4617-9047-7056a320e0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:07.293766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a738473f-42b0-4617-9047-7056a320e0e8 admin admin] PATH_INFO: `/v3/users/797a9311ba0748299c7c9e6e6e3a1e98` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:07.295041 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738473f-42b0-4617-9047-7056a320e0e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=797a9311ba0748299c7c9e6e6e3a1e98)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:07.296974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a738473f-42b0-4617-9047-7056a320e0e8 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:07.332573 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8fa1ef-5e71-4113-9a80-f0e7685e6d5c admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:07.338347 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b571055-3c6d-435d-9592-bace22b8bce2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:07.338620 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:07.339264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b571055-3c6d-435d-9592-bace22b8bce2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:07.339552 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b571055-3c6d-435d-9592-bace22b8bce2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:07.340046 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b571055-3c6d-435d-9592-bace22b8bce2 admin admin] PATH_INFO: `/v3/users/b64dc90488534a8d9e0e2e253456156b` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:07.341173 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b571055-3c6d-435d-9592-bace22b8bce2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b64dc90488534a8d9e0e2e253456156b)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:07.343142 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b571055-3c6d-435d-9592-bace22b8bce2 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:07.349010 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-a738473f-42b0-4617-9047-7056a320e0e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '797a9311ba0748299c7c9e6e6e3a1e98'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.349677 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-a738473f-42b0-4617-9047-7056a320e0e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '797a9311ba0748299c7c9e6e6e3a1e98'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.356892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-a738473f-42b0-4617-9047-7056a320e0e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '797a9311ba0748299c7c9e6e6e3a1e98'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.357990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-a738473f-42b0-4617-9047-7056a320e0e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '797a9311ba0748299c7c9e6e6e3a1e98'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.372213 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-a738473f-42b0-4617-9047-7056a320e0e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '797a9311ba0748299c7c9e6e6e3a1e98'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.387181 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1312/2642] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:07 2022] DELETE /identity/v3/users/797a9311ba0748299c7c9e6e6e3a1e98 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:07.392797 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5b571055-3c6d-435d-9592-bace22b8bce2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b64dc90488534a8d9e0e2e253456156b'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.400563 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5b571055-3c6d-435d-9592-bace22b8bce2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b64dc90488534a8d9e0e2e253456156b'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.401273 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5b571055-3c6d-435d-9592-bace22b8bce2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b64dc90488534a8d9e0e2e253456156b'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.419562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5b571055-3c6d-435d-9592-bace22b8bce2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b64dc90488534a8d9e0e2e253456156b'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.419562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5b571055-3c6d-435d-9592-bace22b8bce2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b64dc90488534a8d9e0e2e253456156b'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.432129 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1331/2643] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:07 2022] DELETE /identity/v3/users/b64dc90488534a8d9e0e2e253456156b => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:07.437364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b6c7c-a268-4dbb-a384-6d44113a0a83 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:07.438225 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b6c7c-a268-4dbb-a384-6d44113a0a83 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:07.438509 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727b6c7c-a268-4dbb-a384-6d44113a0a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:07.457878 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-727b6c7c-a268-4dbb-a384-6d44113a0a83 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:07.459383 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-727b6c7c-a268-4dbb-a384-6d44113a0a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:07.540652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1313/2644] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:07.556393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b571055-3c6d-435d-9592-bace22b8bce2 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:07.558369 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5b571055-3c6d-435d-9592-bace22b8bce2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:07.629359 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:07.636968 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b571055-3c6d-435d-9592-bace22b8bce2 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:07.642181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f21da62-4cb5-4b19-8f77-fffcc3fe4576 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:07.643497 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21da62-4cb5-4b19-8f77-fffcc3fe4576 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:07.643811 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21da62-4cb5-4b19-8f77-fffcc3fe4576 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:07.645100 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f21da62-4cb5-4b19-8f77-fffcc3fe4576 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:07.657625 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f21da62-4cb5-4b19-8f77-fffcc3fe4576 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:07.659287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f21da62-4cb5-4b19-8f77-fffcc3fe4576 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:07.672848 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1332/2645] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:07.756825 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b6c7c-a268-4dbb-a384-6d44113a0a83 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:07.759191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-727b6c7c-a268-4dbb-a384-6d44113a0a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:07.774833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727b6c7c-a268-4dbb-a384-6d44113a0a83 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:07.779725 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57c2f7b-d5e2-4e60-b2bb-a4d84756f0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:07.780599 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c2f7b-d5e2-4e60-b2bb-a4d84756f0bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:07.781053 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c2f7b-d5e2-4e60-b2bb-a4d84756f0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:07.781343 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57c2f7b-d5e2-4e60-b2bb-a4d84756f0bc admin admin] PATH_INFO: `/v3/projects/5d260517390f436495b4bc5e4e356728` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:07.782665 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c2f7b-d5e2-4e60-b2bb-a4d84756f0bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d260517390f436495b4bc5e4e356728)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:07.784617 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57c2f7b-d5e2-4e60-b2bb-a4d84756f0bc admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:07.884488 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-a57c2f7b-d5e2-4e60-b2bb-a4d84756f0bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d260517390f436495b4bc5e4e356728', 'request_id': 'req-a57c2f7b-d5e2-4e60-b2bb-a4d84756f0bc'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.890572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-a57c2f7b-d5e2-4e60-b2bb-a4d84756f0bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d260517390f436495b4bc5e4e356728', 'request_id': 'req-a57c2f7b-d5e2-4e60-b2bb-a4d84756f0bc'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.896345 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-a57c2f7b-d5e2-4e60-b2bb-a4d84756f0bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d260517390f436495b4bc5e4e356728', 'request_id': 'req-a57c2f7b-d5e2-4e60-b2bb-a4d84756f0bc'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.911247 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-a57c2f7b-d5e2-4e60-b2bb-a4d84756f0bc admin admin] The token cache is being invalidate because project 5d260517390f436495b4bc5e4e356728 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:07.911592 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-a57c2f7b-d5e2-4e60-b2bb-a4d84756f0bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.912238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-a57c2f7b-d5e2-4e60-b2bb-a4d84756f0bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:07.913919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1314/2646] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:07 2022] DELETE /identity/v3/projects/5d260517390f436495b4bc5e4e356728 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:07.918903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bed15b-1f8c-4ad7-a33c-7496b0e288c5 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:07.919382 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bed15b-1f8c-4ad7-a33c-7496b0e288c5 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:07.919478 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bed15b-1f8c-4ad7-a33c-7496b0e288c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:07.949083 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-22bed15b-1f8c-4ad7-a33c-7496b0e288c5 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:07.950484 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-22bed15b-1f8c-4ad7-a33c-7496b0e288c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:08.012421 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1333/2647] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:08.020220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb0db1-6ec4-4efc-b254-3c1c84bd0e62 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:08.020509 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb0db1-6ec4-4efc-b254-3c1c84bd0e62 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:08.020785 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87eb0db1-6ec4-4efc-b254-3c1c84bd0e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:08.041770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-87eb0db1-6ec4-4efc-b254-3c1c84bd0e62 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:08.041770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-87eb0db1-6ec4-4efc-b254-3c1c84bd0e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:08.043692 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bed15b-1f8c-4ad7-a33c-7496b0e288c5 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:08.045219 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-22bed15b-1f8c-4ad7-a33c-7496b0e288c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:08.050267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1315/2648] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:08.054037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb0db1-6ec4-4efc-b254-3c1c84bd0e62 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:08.059262 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bed15b-1f8c-4ad7-a33c-7496b0e288c5 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:08.065513 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87eb0db1-6ec4-4efc-b254-3c1c84bd0e62 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:08.068412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd8042-a2f5-47c3-8c1b-b0379c7150bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:08.069724 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd8042-a2f5-47c3-8c1b-b0379c7150bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:08.070519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd8042-a2f5-47c3-8c1b-b0379c7150bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:08.070835 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cd8042-a2f5-47c3-8c1b-b0379c7150bf admin admin] PATH_INFO: `/v3/projects/fb1e8a75367d44eb9df63c5b20c50454` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:08.076028 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd8042-a2f5-47c3-8c1b-b0379c7150bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb1e8a75367d44eb9df63c5b20c50454)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:08.076130 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201fcf6-2788-48e0-91e8-74fd40bc2def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:08.076232 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cd8042-a2f5-47c3-8c1b-b0379c7150bf admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:08.076320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201fcf6-2788-48e0-91e8-74fd40bc2def admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:08.076320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201fcf6-2788-48e0-91e8-74fd40bc2def admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:08.076320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3201fcf6-2788-48e0-91e8-74fd40bc2def admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:08.076320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201fcf6-2788-48e0-91e8-74fd40bc2def admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:08:08.076592 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201fcf6-2788-48e0-91e8-74fd40bc2def admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:08.078337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3201fcf6-2788-48e0-91e8-74fd40bc2def admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:08.091904 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1316/2649] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:08:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:08:08.096141 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201fcf6-2788-48e0-91e8-74fd40bc2def admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:08.109147 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3201fcf6-2788-48e0-91e8-74fd40bc2def admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:08.116383 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d9bbe-9f58-4beb-a30e-8c8d39d80f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:08.117693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d9bbe-9f58-4beb-a30e-8c8d39d80f9b admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:08.117981 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d9bbe-9f58-4beb-a30e-8c8d39d80f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:08.118381 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d9bbe-9f58-4beb-a30e-8c8d39d80f9b admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:08.119159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d9bbe-9f58-4beb-a30e-8c8d39d80f9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:08.121339 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d9bbe-9f58-4beb-a30e-8c8d39d80f9b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:08.131742 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e5cd8042-a2f5-47c3-8c1b-b0379c7150bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb1e8a75367d44eb9df63c5b20c50454', 'request_id': 'req-e5cd8042-a2f5-47c3-8c1b-b0379c7150bf'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:08.139897 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e5cd8042-a2f5-47c3-8c1b-b0379c7150bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb1e8a75367d44eb9df63c5b20c50454', 'request_id': 'req-e5cd8042-a2f5-47c3-8c1b-b0379c7150bf'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:08.140417 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:08.147058 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e5cd8042-a2f5-47c3-8c1b-b0379c7150bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb1e8a75367d44eb9df63c5b20c50454', 'request_id': 'req-e5cd8042-a2f5-47c3-8c1b-b0379c7150bf'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:08.161251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1317/2650] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:08 2022] POST /identity/v3/projects => generated 367 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:08:08.166083 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d9bbe-9f58-4beb-a30e-8c8d39d80f9b admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:08.166987 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e5cd8042-a2f5-47c3-8c1b-b0379c7150bf admin admin] The token cache is being invalidate because project fb1e8a75367d44eb9df63c5b20c50454 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:08.167287 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e5cd8042-a2f5-47c3-8c1b-b0379c7150bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:08.167957 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-e5cd8042-a2f5-47c3-8c1b-b0379c7150bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:08.169520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1334/2651] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:08 2022] DELETE /identity/v3/projects/fb1e8a75367d44eb9df63c5b20c50454 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:08.173217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd8042-a2f5-47c3-8c1b-b0379c7150bf admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:08.174537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-e5cd8042-a2f5-47c3-8c1b-b0379c7150bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:08.238045 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d9bbe-9f58-4beb-a30e-8c8d39d80f9b admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:08.245921 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8980d-6f1d-4cc1-a36b-d434c7baf145 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:08.247082 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8980d-6f1d-4cc1-a36b-d434c7baf145 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:08.247448 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8980d-6f1d-4cc1-a36b-d434c7baf145 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:08.247794 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d8980d-6f1d-4cc1-a36b-d434c7baf145 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:08.248543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8980d-6f1d-4cc1-a36b-d434c7baf145 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:08.251207 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d8980d-6f1d-4cc1-a36b-d434c7baf145 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:08.255910 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cd8042-a2f5-47c3-8c1b-b0379c7150bf admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:08.261127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ff1df6-c01d-4977-a60e-8d76490a363d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:08.262197 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff1df6-c01d-4977-a60e-8d76490a363d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:08.262583 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff1df6-c01d-4977-a60e-8d76490a363d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:08.262994 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ff1df6-c01d-4977-a60e-8d76490a363d admin admin] PATH_INFO: `/v3/users/89c41da62fc04f5094f8a15ee9d7ba5a` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:08.264412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff1df6-c01d-4977-a60e-8d76490a363d admin admin] RBAC: Authorizing `identity:delete_user(user_id=89c41da62fc04f5094f8a15ee9d7ba5a)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:08.266575 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ff1df6-c01d-4977-a60e-8d76490a363d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:08.273020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1318/2652] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:08 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:08:08.277636 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8980d-6f1d-4cc1-a36b-d434c7baf145 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:08.279793 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f3d8980d-6f1d-4cc1-a36b-d434c7baf145 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:08.286794 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:08.293959 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d8980d-6f1d-4cc1-a36b-d434c7baf145 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:08.299193 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a619928-bc77-4f49-acdf-dc2c03a1d532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:08.300015 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a619928-bc77-4f49-acdf-dc2c03a1d532 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:08.300305 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a619928-bc77-4f49-acdf-dc2c03a1d532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:08.300673 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a619928-bc77-4f49-acdf-dc2c03a1d532 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:08.301318 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a619928-bc77-4f49-acdf-dc2c03a1d532 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:08.303020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a619928-bc77-4f49-acdf-dc2c03a1d532 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:08.316903 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1319/2653] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:08:08.317917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a619928-bc77-4f49-acdf-dc2c03a1d532 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:08.325543 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-25ff1df6-c01d-4977-a60e-8d76490a363d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89c41da62fc04f5094f8a15ee9d7ba5a'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:08.331370 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a619928-bc77-4f49-acdf-dc2c03a1d532 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:08.333262 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-25ff1df6-c01d-4977-a60e-8d76490a363d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89c41da62fc04f5094f8a15ee9d7ba5a'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:08.334082 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-25ff1df6-c01d-4977-a60e-8d76490a363d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89c41da62fc04f5094f8a15ee9d7ba5a'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:08.337424 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbe2ba-104f-45d3-b924-79f6f816b12a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:08.338102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbe2ba-104f-45d3-b924-79f6f816b12a admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:08.338394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbe2ba-104f-45d3-b924-79f6f816b12a admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:08.338720 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecbe2ba-104f-45d3-b924-79f6f816b12a admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:08.339446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbe2ba-104f-45d3-b924-79f6f816b12a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:08.341064 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecbe2ba-104f-45d3-b924-79f6f816b12a admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:08.347769 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-25ff1df6-c01d-4977-a60e-8d76490a363d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89c41da62fc04f5094f8a15ee9d7ba5a'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:08.348478 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-25ff1df6-c01d-4977-a60e-8d76490a363d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89c41da62fc04f5094f8a15ee9d7ba5a'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:08.354666 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1320/2654] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:08:08.357210 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbe2ba-104f-45d3-b924-79f6f816b12a admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:08.358946 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-2ecbe2ba-104f-45d3-b924-79f6f816b12a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:08.362192 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1335/2655] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:08 2022] DELETE /identity/v3/users/89c41da62fc04f5094f8a15ee9d7ba5a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:08.372476 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecbe2ba-104f-45d3-b924-79f6f816b12a admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:08.433696 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444546b-e491-4927-be17-d6148fdd2a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:08.434614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444546b-e491-4927-be17-d6148fdd2a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:08.434923 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444546b-e491-4927-be17-d6148fdd2a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:08.435204 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8444546b-e491-4927-be17-d6148fdd2a92 admin admin] PATH_INFO: `/v3/projects/478212b6dc6042268691207f6e3d957e/users/068d4e0774704d86afd61e1d18212101/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:08.454656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444546b-e491-4927-be17-d6148fdd2a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=478212b6dc6042268691207f6e3d957e, user_id=068d4e0774704d86afd61e1d18212101, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:08.454656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8444546b-e491-4927-be17-d6148fdd2a92 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:08.468574 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1321/2656] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:08 2022] PUT /identity/v3/projects/478212b6dc6042268691207f6e3d957e/users/068d4e0774704d86afd61e1d18212101/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:08.479492 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe6538-d9c5-460a-84b5-bd5644536b2f None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:08.479790 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe6538-d9c5-460a-84b5-bd5644536b2f None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:08.480075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fe6538-d9c5-460a-84b5-bd5644536b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:08.522457 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-74fe6538-d9c5-460a-84b5-bd5644536b2f None None] MFA Rules not processed for user `068d4e0774704d86afd61e1d18212101`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:08.525003 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-74fe6538-d9c5-460a-84b5-bd5644536b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:08.602642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1336/2657] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:08 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:08.613006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444546b-e491-4927-be17-d6148fdd2a92 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:08.614776 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-8444546b-e491-4927-be17-d6148fdd2a92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:08.651188 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe6538-d9c5-460a-84b5-bd5644536b2f None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:08.652891 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-74fe6538-d9c5-460a-84b5-bd5644536b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:08.683784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:08.690319 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8444546b-e491-4927-be17-d6148fdd2a92 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:08.695379 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde06ec4-6cf0-46a5-9f42-6acad2711115 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915f4296504b47d796e3a5b99550eb75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:08.696190 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde06ec4-6cf0-46a5-9f42-6acad2711115 service glance] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:08.696733 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde06ec4-6cf0-46a5-9f42-6acad2711115 service glance] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:08.697013 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde06ec4-6cf0-46a5-9f42-6acad2711115 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:08.705908 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde06ec4-6cf0-46a5-9f42-6acad2711115 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:08.707382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde06ec4-6cf0-46a5-9f42-6acad2711115 service glance] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:08.716994 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1322/2658] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:08 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:08.749746 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fe6538-d9c5-460a-84b5-bd5644536b2f None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:08.755226 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f430f88-57a6-4154-a914-2ebf79e82fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:08.756094 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f430f88-57a6-4154-a914-2ebf79e82fb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:08.756384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f430f88-57a6-4154-a914-2ebf79e82fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:08.756664 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f430f88-57a6-4154-a914-2ebf79e82fb6 admin admin] PATH_INFO: `/v3/projects/b1b665b659c947f79983ee60a84dffae` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:08.758695 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f430f88-57a6-4154-a914-2ebf79e82fb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1b665b659c947f79983ee60a84dffae)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:08.760714 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f430f88-57a6-4154-a914-2ebf79e82fb6 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:08.789450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:08.823272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8f430f88-57a6-4154-a914-2ebf79e82fb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1b665b659c947f79983ee60a84dffae', 'request_id': 'req-8f430f88-57a6-4154-a914-2ebf79e82fb6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:08.833107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8f430f88-57a6-4154-a914-2ebf79e82fb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1b665b659c947f79983ee60a84dffae', 'request_id': 'req-8f430f88-57a6-4154-a914-2ebf79e82fb6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:08.840502 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8f430f88-57a6-4154-a914-2ebf79e82fb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1b665b659c947f79983ee60a84dffae', 'request_id': 'req-8f430f88-57a6-4154-a914-2ebf79e82fb6'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:08.861410 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8f430f88-57a6-4154-a914-2ebf79e82fb6 admin admin] The token cache is being invalidate because project b1b665b659c947f79983ee60a84dffae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:08.861710 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8f430f88-57a6-4154-a914-2ebf79e82fb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:08.862375 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8f430f88-57a6-4154-a914-2ebf79e82fb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:08.864159 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1337/2659] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:08 2022] DELETE /identity/v3/projects/b1b665b659c947f79983ee60a84dffae => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:08.868976 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e45a64-3cf6-48f9-bbe2-1a8ab3c02a10 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:08.869335 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e45a64-3cf6-48f9-bbe2-1a8ab3c02a10 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:08.869686 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e45a64-3cf6-48f9-bbe2-1a8ab3c02a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:08.892258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-01e45a64-3cf6-48f9-bbe2-1a8ab3c02a10 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:08.893496 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-01e45a64-3cf6-48f9-bbe2-1a8ab3c02a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:08.986278 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1323/2660] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:08.996451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687515a-876a-4c20-b7cc-1debf9370b7f None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:08.997974 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687515a-876a-4c20-b7cc-1debf9370b7f None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:08.997974 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a687515a-876a-4c20-b7cc-1debf9370b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:09.017075 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a687515a-876a-4c20-b7cc-1debf9370b7f None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:09.018213 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a687515a-876a-4c20-b7cc-1debf9370b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:09.034625 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1338/2661] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:09.035813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e45a64-3cf6-48f9-bbe2-1a8ab3c02a10 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:09.046158 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e45a64-3cf6-48f9-bbe2-1a8ab3c02a10 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:09.052373 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668adfe-9c05-460d-9ebe-d51c5f48bec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:09.053105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668adfe-9c05-460d-9ebe-d51c5f48bec2 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:09.053356 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668adfe-9c05-460d-9ebe-d51c5f48bec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:09.053810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f668adfe-9c05-460d-9ebe-d51c5f48bec2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:09.054955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668adfe-9c05-460d-9ebe-d51c5f48bec2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:08:09.055259 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668adfe-9c05-460d-9ebe-d51c5f48bec2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:09.056875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f668adfe-9c05-460d-9ebe-d51c5f48bec2 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:09.078967 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1324/2662] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:08:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:08:09.079135 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687515a-876a-4c20-b7cc-1debf9370b7f None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:09.089306 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a687515a-876a-4c20-b7cc-1debf9370b7f None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:09.099412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bc6f2-5658-40b2-ac92-294655773f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:09.100329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bc6f2-5658-40b2-ac92-294655773f8f admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:09.101015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bc6f2-5658-40b2-ac92-294655773f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:09.102046 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bc6f2-5658-40b2-ac92-294655773f8f admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:09.102933 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bc6f2-5658-40b2-ac92-294655773f8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:09.105598 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bc6f2-5658-40b2-ac92-294655773f8f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:09.149056 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668adfe-9c05-460d-9ebe-d51c5f48bec2 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:09.152021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-f668adfe-9c05-460d-9ebe-d51c5f48bec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:09.158653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1339/2663] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:09 2022] POST /identity/v3/projects => generated 381 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:09.158653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bc6f2-5658-40b2-ac92-294655773f8f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:09.244790 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f668adfe-9c05-460d-9ebe-d51c5f48bec2 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:09.254773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bc6f2-5658-40b2-ac92-294655773f8f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:09.260393 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d58972-f6ae-4db8-b50a-b87775a10c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:09.261451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d58972-f6ae-4db8-b50a-b87775a10c7a admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:09.261936 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d58972-f6ae-4db8-b50a-b87775a10c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:09.262440 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d58972-f6ae-4db8-b50a-b87775a10c7a admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:09.263320 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d58972-f6ae-4db8-b50a-b87775a10c7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:09.265749 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d58972-f6ae-4db8-b50a-b87775a10c7a admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:09.292643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1340/2664] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:09 2022] POST /identity/v3/users => generated 291 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:09.305682 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d58972-f6ae-4db8-b50a-b87775a10c7a admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:09.312932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d58972-f6ae-4db8-b50a-b87775a10c7a admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:09.318255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c981649-d871-4fbd-8c73-c59256302e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:09.319098 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c981649-d871-4fbd-8c73-c59256302e4b admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:09.319655 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c981649-d871-4fbd-8c73-c59256302e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:09.320109 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c981649-d871-4fbd-8c73-c59256302e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:09.321604 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c981649-d871-4fbd-8c73-c59256302e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:09.327697 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c981649-d871-4fbd-8c73-c59256302e4b admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:09.340422 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bb72c-ca0e-447a-b992-1642ca25498c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:09.341906 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bb72c-ca0e-447a-b992-1642ca25498c service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:09.342380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bb72c-ca0e-447a-b992-1642ca25498c service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:09.342864 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17bb72c-ca0e-447a-b992-1642ca25498c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:09.343654 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1341/2665] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:09.344651 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c981649-d871-4fbd-8c73-c59256302e4b admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:09.345426 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d17bb72c-ca0e-447a-b992-1642ca25498c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:09.357424 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c981649-d871-4fbd-8c73-c59256302e4b admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:09.360774 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a000df-a6f2-4ea0-9b2c-36aea7680d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:09.361929 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a000df-a6f2-4ea0-9b2c-36aea7680d7d admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:09.362700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a000df-a6f2-4ea0-9b2c-36aea7680d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:09.362983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a000df-a6f2-4ea0-9b2c-36aea7680d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:09.363810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a000df-a6f2-4ea0-9b2c-36aea7680d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:09.365476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a000df-a6f2-4ea0-9b2c-36aea7680d7d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:09.378712 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1342/2666] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:09.382805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a000df-a6f2-4ea0-9b2c-36aea7680d7d admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:09.393336 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a000df-a6f2-4ea0-9b2c-36aea7680d7d admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:09.398535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3085733-8236-4eed-95c0-37367d101faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:09.399456 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3085733-8236-4eed-95c0-37367d101faf admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:09.399741 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3085733-8236-4eed-95c0-37367d101faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:09.400017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3085733-8236-4eed-95c0-37367d101faf admin admin] PATH_INFO: `/v3/projects/48cfad6adf9c4e76995345e71d84fb2a/users/21e9e8ae95934c7bb853756568f67063/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:09.416291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:09.416516 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3085733-8236-4eed-95c0-37367d101faf admin admin] RBAC: Authorizing `identity:create_grant(project_id=48cfad6adf9c4e76995345e71d84fb2a, user_id=21e9e8ae95934c7bb853756568f67063, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:09.421310 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3085733-8236-4eed-95c0-37367d101faf admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:09.433616 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17bb72c-ca0e-447a-b992-1642ca25498c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:09.435424 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d17bb72c-ca0e-447a-b992-1642ca25498c service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:09.436915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1343/2667] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:09 2022] PUT /identity/v3/projects/48cfad6adf9c4e76995345e71d84fb2a/users/21e9e8ae95934c7bb853756568f67063/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:09.445306 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1325/2668] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:08:09.452200 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af3b73-f7d6-4783-acee-42c136f59128 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:09.452783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af3b73-f7d6-4783-acee-42c136f59128 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:09.453150 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73af3b73-f7d6-4783-acee-42c136f59128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:09.491102 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-73af3b73-f7d6-4783-acee-42c136f59128 None None] MFA Rules not processed for user `21e9e8ae95934c7bb853756568f67063`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:09.492826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-73af3b73-f7d6-4783-acee-42c136f59128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:09.506894 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=88,keystone:DELETE=6,keystone:INSERT=6 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:09.568653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1344/2669] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:09.574340 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bb72c-ca0e-447a-b992-1642ca25498c service cinder] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:09.588166 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d17bb72c-ca0e-447a-b992-1642ca25498c service cinder] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:09.660986 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c58278-bac9-4a18-9965-4d8b8ca25eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:09.661918 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c58278-bac9-4a18-9965-4d8b8ca25eb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:09.662211 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c58278-bac9-4a18-9965-4d8b8ca25eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:09.662494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c58278-bac9-4a18-9965-4d8b8ca25eb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:09.663244 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c58278-bac9-4a18-9965-4d8b8ca25eb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:09.666285 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c58278-bac9-4a18-9965-4d8b8ca25eb9 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:09.702086 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1326/2670] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:09 2022] POST /identity/v3/projects => generated 381 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:09.703456 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af3b73-f7d6-4783-acee-42c136f59128 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:09.716071 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73af3b73-f7d6-4783-acee-42c136f59128 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:09.779466 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b898a-0274-466a-af9f-5e2584fb4276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:09.780246 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b898a-0274-466a-af9f-5e2584fb4276 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:09.780544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b898a-0274-466a-af9f-5e2584fb4276 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:09.780828 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281b898a-0274-466a-af9f-5e2584fb4276 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:09.781534 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b898a-0274-466a-af9f-5e2584fb4276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:09.783520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-281b898a-0274-466a-af9f-5e2584fb4276 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:09.807495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1345/2671] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:09 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:09.808729 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c58278-bac9-4a18-9965-4d8b8ca25eb9 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:09.824903 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c58278-bac9-4a18-9965-4d8b8ca25eb9 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:09.830799 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b83f1a-ad7d-4243-abbf-beeb4edf5c08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:09.831510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b83f1a-ad7d-4243-abbf-beeb4edf5c08 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:09.831775 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b83f1a-ad7d-4243-abbf-beeb4edf5c08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:09.832084 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b83f1a-ad7d-4243-abbf-beeb4edf5c08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:09.832794 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b83f1a-ad7d-4243-abbf-beeb4edf5c08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:09.834407 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b83f1a-ad7d-4243-abbf-beeb4edf5c08 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:09.853901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1327/2672] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:09.855141 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b898a-0274-466a-af9f-5e2584fb4276 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:09.865159 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-281b898a-0274-466a-af9f-5e2584fb4276 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:09.870398 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205dc7c5-3dba-4076-bd68-478e4bcf61ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:09.871332 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dc7c5-3dba-4076-bd68-478e4bcf61ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:09.871621 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dc7c5-3dba-4076-bd68-478e4bcf61ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:09.871901 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205dc7c5-3dba-4076-bd68-478e4bcf61ee admin admin] PATH_INFO: `/v3/projects/04d3c9e94d2f4e83a269b093ac2b318a/users/47c4b0d434d34207a36d59d12f0072d2/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:09.889749 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205dc7c5-3dba-4076-bd68-478e4bcf61ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=04d3c9e94d2f4e83a269b093ac2b318a, user_id=47c4b0d434d34207a36d59d12f0072d2, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:09.892781 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205dc7c5-3dba-4076-bd68-478e4bcf61ee admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:09.910207 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1346/2673] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:09 2022] PUT /identity/v3/projects/04d3c9e94d2f4e83a269b093ac2b318a/users/47c4b0d434d34207a36d59d12f0072d2/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:09.914480 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b83f1a-ad7d-4243-abbf-beeb4edf5c08 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:09.928807 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b83f1a-ad7d-4243-abbf-beeb4edf5c08 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:09.988181 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636fe8e-5162-47ed-acaf-15cce345a2df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:09.988861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636fe8e-5162-47ed-acaf-15cce345a2df admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:09.989148 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636fe8e-5162-47ed-acaf-15cce345a2df admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:09.989426 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636fe8e-5162-47ed-acaf-15cce345a2df admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:09.990150 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636fe8e-5162-47ed-acaf-15cce345a2df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:09.992417 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636fe8e-5162-47ed-acaf-15cce345a2df admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:10.005174 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1328/2674] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:09 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:10.006342 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205dc7c5-3dba-4076-bd68-478e4bcf61ee admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:10.017346 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205dc7c5-3dba-4076-bd68-478e4bcf61ee admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:10.022444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bb771-fa5e-4225-a307-4840a33080b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:10.023353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bb771-fa5e-4225-a307-4840a33080b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:10.023722 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bb771-fa5e-4225-a307-4840a33080b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:10.023939 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6bb771-fa5e-4225-a307-4840a33080b1 admin admin] PATH_INFO: `/v3/projects/04d3c9e94d2f4e83a269b093ac2b318a/users/47c4b0d434d34207a36d59d12f0072d2/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:10.028154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bb771-fa5e-4225-a307-4840a33080b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04d3c9e94d2f4e83a269b093ac2b318a, user_id=47c4b0d434d34207a36d59d12f0072d2, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:10.031666 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6bb771-fa5e-4225-a307-4840a33080b1 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:10.046002 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1347/2675] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:10 2022] PUT /identity/v3/projects/04d3c9e94d2f4e83a269b093ac2b318a/users/47c4b0d434d34207a36d59d12f0072d2/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:10.058337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe74096-423b-40d5-b6ad-9f8aad1d9bdd None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:10.058572 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe74096-423b-40d5-b6ad-9f8aad1d9bdd None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:10.058874 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe74096-423b-40d5-b6ad-9f8aad1d9bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:10.094449 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-afe74096-423b-40d5-b6ad-9f8aad1d9bdd None None] MFA Rules not processed for user `47c4b0d434d34207a36d59d12f0072d2`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:10.095698 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-afe74096-423b-40d5-b6ad-9f8aad1d9bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:10.143256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:10.175492 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1329/2676] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:10 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:11.613238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bb771-fa5e-4225-a307-4840a33080b1 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:11.615051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ef6bb771-fa5e-4225-a307-4840a33080b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:11.642410 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe74096-423b-40d5-b6ad-9f8aad1d9bdd None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:11.644212 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-afe74096-423b-40d5-b6ad-9f8aad1d9bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:11.683361 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6bb771-fa5e-4225-a307-4840a33080b1 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:11.688369 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3592f9-8a3e-4683-8791-39a9f354a8ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:11.689322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3592f9-8a3e-4683-8791-39a9f354a8ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:11.689616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3592f9-8a3e-4683-8791-39a9f354a8ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:11.689898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3592f9-8a3e-4683-8791-39a9f354a8ed admin admin] PATH_INFO: `/v3/users/63ddf828047548949f85d80807e74b4d` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:11.691132 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3592f9-8a3e-4683-8791-39a9f354a8ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=63ddf828047548949f85d80807e74b4d)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:11.693050 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3592f9-8a3e-4683-8791-39a9f354a8ed admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:11.705094 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe74096-423b-40d5-b6ad-9f8aad1d9bdd None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:11.706935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:11.709327 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e1ac74-1464-4e5f-acd3-a179092c7e16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:11.710022 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1ac74-1464-4e5f-acd3-a179092c7e16 service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:11.710338 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1ac74-1464-4e5f-acd3-a179092c7e16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:11.710661 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e1ac74-1464-4e5f-acd3-a179092c7e16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:11.721562 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1ac74-1464-4e5f-acd3-a179092c7e16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:11.723428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e1ac74-1464-4e5f-acd3-a179092c7e16 service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:11.736604 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1330/2677] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:11 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:08:11.751834 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8b3592f9-8a3e-4683-8791-39a9f354a8ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '63ddf828047548949f85d80807e74b4d'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:11.760644 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8b3592f9-8a3e-4683-8791-39a9f354a8ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '63ddf828047548949f85d80807e74b4d'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:11.761376 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8b3592f9-8a3e-4683-8791-39a9f354a8ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '63ddf828047548949f85d80807e74b4d'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:11.777901 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8b3592f9-8a3e-4683-8791-39a9f354a8ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '63ddf828047548949f85d80807e74b4d'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:11.777901 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-8b3592f9-8a3e-4683-8791-39a9f354a8ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '63ddf828047548949f85d80807e74b4d'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:11.791608 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1348/2678] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:11 2022] DELETE /identity/v3/users/63ddf828047548949f85d80807e74b4d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:11.796054 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38503554-91e8-4bb4-9283-deedc4f44b2c None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:11.796346 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38503554-91e8-4bb4-9283-deedc4f44b2c None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:11.796632 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38503554-91e8-4bb4-9283-deedc4f44b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:11.818203 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-38503554-91e8-4bb4-9283-deedc4f44b2c None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:11.819557 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-38503554-91e8-4bb4-9283-deedc4f44b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:11.891289 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1331/2679] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:11.900443 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3592f9-8a3e-4683-8791-39a9f354a8ed admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:11.901862 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-8b3592f9-8a3e-4683-8791-39a9f354a8ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:11.968536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3592f9-8a3e-4683-8791-39a9f354a8ed admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:11.972506 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb10c23-a1a5-4332-9ef3-5ab5977a2635 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:11.973127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb10c23-a1a5-4332-9ef3-5ab5977a2635 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:11.973482 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb10c23-a1a5-4332-9ef3-5ab5977a2635 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:11.973827 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb10c23-a1a5-4332-9ef3-5ab5977a2635 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:11.984165 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb10c23-a1a5-4332-9ef3-5ab5977a2635 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:11.985786 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb10c23-a1a5-4332-9ef3-5ab5977a2635 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:11.995915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1349/2680] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:12.060408 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38503554-91e8-4bb4-9283-deedc4f44b2c None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:12.061983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-38503554-91e8-4bb4-9283-deedc4f44b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:12.142315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38503554-91e8-4bb4-9283-deedc4f44b2c None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:12.147191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74c54c-0270-4bc3-a7cb-2641dee360b1 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47c4b0d434d34207a36d59d12f0072d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04d3c9e94d2f4e83a269b093ac2b318a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:12.147913 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74c54c-0270-4bc3-a7cb-2641dee360b1 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:12.148313 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74c54c-0270-4bc3-a7cb-2641dee360b1 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:12.148568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a74c54c-0270-4bc3-a7cb-2641dee360b1 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:12.149204 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74c54c-0270-4bc3-a7cb-2641dee360b1 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:08:12.149516 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74c54c-0270-4bc3-a7cb-2641dee360b1 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:12.153606 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a74c54c-0270-4bc3-a7cb-2641dee360b1 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:12.166194 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1332/2681] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:08:12.171072 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb10c23-a1a5-4332-9ef3-5ab5977a2635 service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:12.182143 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb10c23-a1a5-4332-9ef3-5ab5977a2635 service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:12.186975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d726028-13a5-46f9-956e-ae886fb65765 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47c4b0d434d34207a36d59d12f0072d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04d3c9e94d2f4e83a269b093ac2b318a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:12.187738 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d726028-13a5-46f9-956e-ae886fb65765 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:12.188033 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d726028-13a5-46f9-956e-ae886fb65765 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:12.188322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d726028-13a5-46f9-956e-ae886fb65765 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:12.189052 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d726028-13a5-46f9-956e-ae886fb65765 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:12.191099 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d726028-13a5-46f9-956e-ae886fb65765 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:12.228074 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1350/2682] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:12 2022] POST /identity/v3/projects => generated 354 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:08:12.231804 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74c54c-0270-4bc3-a7cb-2641dee360b1 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:12.242772 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a74c54c-0270-4bc3-a7cb-2641dee360b1 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:12.266552 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:12.291585 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d726028-13a5-46f9-956e-ae886fb65765 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:12.293473 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-2d726028-13a5-46f9-956e-ae886fb65765 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:12.332378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038184e-8c18-444b-bcc6-cf6baa85489f tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47c4b0d434d34207a36d59d12f0072d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04d3c9e94d2f4e83a269b093ac2b318a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:12.333067 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038184e-8c18-444b-bcc6-cf6baa85489f tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:12.333363 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038184e-8c18-444b-bcc6-cf6baa85489f tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:12.333651 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e038184e-8c18-444b-bcc6-cf6baa85489f tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:12.334384 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038184e-8c18-444b-bcc6-cf6baa85489f tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:08:12.334739 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038184e-8c18-444b-bcc6-cf6baa85489f tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:12.336372 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e038184e-8c18-444b-bcc6-cf6baa85489f tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:12.361440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1333/2683] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 23:08:12.383517 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d726028-13a5-46f9-956e-ae886fb65765 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:12.388069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e078977-c20a-4c2d-9784-28b31c258f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:12.388930 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e078977-c20a-4c2d-9784-28b31c258f24 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:12.389213 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e078977-c20a-4c2d-9784-28b31c258f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:12.389487 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e078977-c20a-4c2d-9784-28b31c258f24 admin admin] PATH_INFO: `/v3/projects/3f19bd43d6c44eeaa4d0c5c681cebeae` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:12.390683 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e078977-c20a-4c2d-9784-28b31c258f24 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f19bd43d6c44eeaa4d0c5c681cebeae)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:12.397081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e078977-c20a-4c2d-9784-28b31c258f24 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:12.398409 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038184e-8c18-444b-bcc6-cf6baa85489f tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:12.408026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:12.409100 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e038184e-8c18-444b-bcc6-cf6baa85489f tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:12.415536 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6a492-86b4-4b58-bd47-b2c89c415019 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '47c4b0d434d34207a36d59d12f0072d2', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '04d3c9e94d2f4e83a269b093ac2b318a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:12.418494 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6a492-86b4-4b58-bd47-b2c89c415019 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:12.418949 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6a492-86b4-4b58-bd47-b2c89c415019 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:12.419403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b6a492-86b4-4b58-bd47-b2c89c415019 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] PATH_INFO: `/v3/projects/0b10ba7e12f14e0db644e8505178e6e5` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:12.421181 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6a492-86b4-4b58-bd47-b2c89c415019 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0b10ba7e12f14e0db644e8505178e6e5)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:12.423121 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b6a492-86b4-4b58-bd47-b2c89c415019 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:12.459612 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-2e078977-c20a-4c2d-9784-28b31c258f24 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f19bd43d6c44eeaa4d0c5c681cebeae', 'request_id': 'req-2e078977-c20a-4c2d-9784-28b31c258f24'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.467131 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-2e078977-c20a-4c2d-9784-28b31c258f24 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f19bd43d6c44eeaa4d0c5c681cebeae', 'request_id': 'req-2e078977-c20a-4c2d-9784-28b31c258f24'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.473404 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-2e078977-c20a-4c2d-9784-28b31c258f24 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f19bd43d6c44eeaa4d0c5c681cebeae', 'request_id': 'req-2e078977-c20a-4c2d-9784-28b31c258f24'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.481905 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-74b6a492-86b4-4b58-bd47-b2c89c415019 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b10ba7e12f14e0db644e8505178e6e5', 'request_id': 'req-74b6a492-86b4-4b58-bd47-b2c89c415019'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.493119 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-74b6a492-86b4-4b58-bd47-b2c89c415019 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b10ba7e12f14e0db644e8505178e6e5', 'request_id': 'req-74b6a492-86b4-4b58-bd47-b2c89c415019'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.495892 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-2e078977-c20a-4c2d-9784-28b31c258f24 admin admin] The token cache is being invalidate because project 3f19bd43d6c44eeaa4d0c5c681cebeae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:12.496041 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-2e078977-c20a-4c2d-9784-28b31c258f24 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.496735 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-2e078977-c20a-4c2d-9784-28b31c258f24 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.498802 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1351/2684] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:12 2022] DELETE /identity/v3/projects/3f19bd43d6c44eeaa4d0c5c681cebeae => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:12.501983 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-74b6a492-86b4-4b58-bd47-b2c89c415019 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b10ba7e12f14e0db644e8505178e6e5', 'request_id': 'req-74b6a492-86b4-4b58-bd47-b2c89c415019'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.502306 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e078977-c20a-4c2d-9784-28b31c258f24 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:12.503996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-2e078977-c20a-4c2d-9784-28b31c258f24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:12.516607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-74b6a492-86b4-4b58-bd47-b2c89c415019 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] The token cache is being invalidate because project 0b10ba7e12f14e0db644e8505178e6e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:12.517024 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-74b6a492-86b4-4b58-bd47-b2c89c415019 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.517777 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-74b6a492-86b4-4b58-bd47-b2c89c415019 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.519643 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1334/2685] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:12 2022] DELETE /identity/v3/projects/0b10ba7e12f14e0db644e8505178e6e5 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:12.591975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e078977-c20a-4c2d-9784-28b31c258f24 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:12.655736 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f8c11-27c7-4640-b239-c07e312787af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:12.656662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f8c11-27c7-4640-b239-c07e312787af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:12.656973 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f8c11-27c7-4640-b239-c07e312787af admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:12.657212 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f8c11-27c7-4640-b239-c07e312787af admin admin] PATH_INFO: `/v3/users/6e62eaf20c45403791a0f80f34ee4f08` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:12.658410 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f8c11-27c7-4640-b239-c07e312787af admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e62eaf20c45403791a0f80f34ee4f08)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:12.660343 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f8c11-27c7-4640-b239-c07e312787af admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:12.715891 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-f52f8c11-27c7-4640-b239-c07e312787af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e62eaf20c45403791a0f80f34ee4f08'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.717309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:12.722101 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6a492-86b4-4b58-bd47-b2c89c415019 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:12.723955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-74b6a492-86b4-4b58-bd47-b2c89c415019 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:12.728704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-f52f8c11-27c7-4640-b239-c07e312787af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e62eaf20c45403791a0f80f34ee4f08'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.729790 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-f52f8c11-27c7-4640-b239-c07e312787af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e62eaf20c45403791a0f80f34ee4f08'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.739559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b6a492-86b4-4b58-bd47-b2c89c415019 tempest-VolumeQuotaClassesTest-1109956896 tempest-VolumeQuotaClassesTest-1109956896-project-admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:12.744908 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa61d4-5bd8-44dd-a466-4665c1e04f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:12.745945 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa61d4-5bd8-44dd-a466-4665c1e04f32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:12.746229 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa61d4-5bd8-44dd-a466-4665c1e04f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:12.746509 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fa61d4-5bd8-44dd-a466-4665c1e04f32 admin admin] PATH_INFO: `/v3/users/21e9e8ae95934c7bb853756568f67063` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:12.747741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa61d4-5bd8-44dd-a466-4665c1e04f32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21e9e8ae95934c7bb853756568f67063)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:12.747886 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-f52f8c11-27c7-4640-b239-c07e312787af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e62eaf20c45403791a0f80f34ee4f08'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.748514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-f52f8c11-27c7-4640-b239-c07e312787af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e62eaf20c45403791a0f80f34ee4f08'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.749648 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fa61d4-5bd8-44dd-a466-4665c1e04f32 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:12.761026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1352/2686] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:12 2022] DELETE /identity/v3/users/6e62eaf20c45403791a0f80f34ee4f08 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:12.805873 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d8fa61d4-5bd8-44dd-a466-4665c1e04f32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21e9e8ae95934c7bb853756568f67063'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.806540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d8fa61d4-5bd8-44dd-a466-4665c1e04f32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21e9e8ae95934c7bb853756568f67063'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.813770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d8fa61d4-5bd8-44dd-a466-4665c1e04f32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21e9e8ae95934c7bb853756568f67063'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.814528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d8fa61d4-5bd8-44dd-a466-4665c1e04f32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21e9e8ae95934c7bb853756568f67063'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.832025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d8fa61d4-5bd8-44dd-a466-4665c1e04f32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21e9e8ae95934c7bb853756568f67063'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:12.846799 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1335/2687] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:12 2022] DELETE /identity/v3/users/21e9e8ae95934c7bb853756568f67063 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:12.851857 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852f30c-4d66-497c-bfe3-c4e1e375ef5b None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:12.852181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852f30c-4d66-497c-bfe3-c4e1e375ef5b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:12.852461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c852f30c-4d66-497c-bfe3-c4e1e375ef5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:12.874073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-c852f30c-4d66-497c-bfe3-c4e1e375ef5b None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:12.875372 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c852f30c-4d66-497c-bfe3-c4e1e375ef5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:12.948643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1353/2688] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:12.958259 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa61d4-5bd8-44dd-a466-4665c1e04f32 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:12.959726 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d8fa61d4-5bd8-44dd-a466-4665c1e04f32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:12.971741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:13.022398 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fa61d4-5bd8-44dd-a466-4665c1e04f32 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:13.027268 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d6e2e-b9cd-4c27-b470-879a8b1b9348 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:13.027924 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d6e2e-b9cd-4c27-b470-879a8b1b9348 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:13.028210 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d6e2e-b9cd-4c27-b470-879a8b1b9348 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:13.028491 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703d6e2e-b9cd-4c27-b470-879a8b1b9348 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:13.038920 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d6e2e-b9cd-4c27-b470-879a8b1b9348 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:13.040353 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703d6e2e-b9cd-4c27-b470-879a8b1b9348 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:13.051208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1336/2689] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:13.065971 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852f30c-4d66-497c-bfe3-c4e1e375ef5b None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:13.067436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c852f30c-4d66-497c-bfe3-c4e1e375ef5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:13.081782 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c852f30c-4d66-497c-bfe3-c4e1e375ef5b None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:13.090965 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c76f4-881d-4bba-8df3-265c359723eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:13.091969 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c76f4-881d-4bba-8df3-265c359723eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:13.092337 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c76f4-881d-4bba-8df3-265c359723eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:13.092620 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c76f4-881d-4bba-8df3-265c359723eb admin admin] PATH_INFO: `/v3/projects/e578f3aba1d743a3a6fe5e6ae8ab3548` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:13.094704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c76f4-881d-4bba-8df3-265c359723eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=e578f3aba1d743a3a6fe5e6ae8ab3548)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:13.097341 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1c76f4-881d-4bba-8df3-265c359723eb admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:13.158872 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3e1c76f4-881d-4bba-8df3-265c359723eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e578f3aba1d743a3a6fe5e6ae8ab3548', 'request_id': 'req-3e1c76f4-881d-4bba-8df3-265c359723eb'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.166373 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3e1c76f4-881d-4bba-8df3-265c359723eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e578f3aba1d743a3a6fe5e6ae8ab3548', 'request_id': 'req-3e1c76f4-881d-4bba-8df3-265c359723eb'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.172410 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3e1c76f4-881d-4bba-8df3-265c359723eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e578f3aba1d743a3a6fe5e6ae8ab3548', 'request_id': 'req-3e1c76f4-881d-4bba-8df3-265c359723eb'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.184187 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3e1c76f4-881d-4bba-8df3-265c359723eb admin admin] The token cache is being invalidate because project e578f3aba1d743a3a6fe5e6ae8ab3548 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:13.184564 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3e1c76f4-881d-4bba-8df3-265c359723eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.185284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-3e1c76f4-881d-4bba-8df3-265c359723eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.187074 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1354/2690] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:13 2022] DELETE /identity/v3/projects/e578f3aba1d743a3a6fe5e6ae8ab3548 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:13.193985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d6e2e-b9cd-4c27-b470-879a8b1b9348 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:13.203882 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703d6e2e-b9cd-4c27-b470-879a8b1b9348 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:13.273393 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3c819-a57b-4f9c-a442-0d725f7f26a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:13.274291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3c819-a57b-4f9c-a442-0d725f7f26a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:13.274575 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3c819-a57b-4f9c-a442-0d725f7f26a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:13.274884 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3c819-a57b-4f9c-a442-0d725f7f26a4 admin admin] PATH_INFO: `/v3/users/4f2913ad5b8f440a9ec71a507b43fc0f` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:13.276135 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a3c819-a57b-4f9c-a442-0d725f7f26a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f2913ad5b8f440a9ec71a507b43fc0f)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:13.278056 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a3c819-a57b-4f9c-a442-0d725f7f26a4 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:13.343569 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c4a3c819-a57b-4f9c-a442-0d725f7f26a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f2913ad5b8f440a9ec71a507b43fc0f'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.344306 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c4a3c819-a57b-4f9c-a442-0d725f7f26a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f2913ad5b8f440a9ec71a507b43fc0f'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.351945 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c4a3c819-a57b-4f9c-a442-0d725f7f26a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f2913ad5b8f440a9ec71a507b43fc0f'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.352697 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c4a3c819-a57b-4f9c-a442-0d725f7f26a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f2913ad5b8f440a9ec71a507b43fc0f'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.360158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c76f4-881d-4bba-8df3-265c359723eb admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:13.361607 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-3e1c76f4-881d-4bba-8df3-265c359723eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:13.371645 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-c4a3c819-a57b-4f9c-a442-0d725f7f26a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f2913ad5b8f440a9ec71a507b43fc0f'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.375419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c76f4-881d-4bba-8df3-265c359723eb admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:13.388117 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9e6b2-b1fa-4c15-9e78-18f34f064d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:13.389796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9e6b2-b1fa-4c15-9e78-18f34f064d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:13.390088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9e6b2-b1fa-4c15-9e78-18f34f064d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:13.390433 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b9e6b2-b1fa-4c15-9e78-18f34f064d01 admin admin] PATH_INFO: `/v3/projects/48cfad6adf9c4e76995345e71d84fb2a` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:13.392407 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9e6b2-b1fa-4c15-9e78-18f34f064d01 admin admin] RBAC: Authorizing `identity:delete_project(project_id=48cfad6adf9c4e76995345e71d84fb2a)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:13.393836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1337/2691] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:13 2022] DELETE /identity/v3/users/4f2913ad5b8f440a9ec71a507b43fc0f => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:13.394435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b9e6b2-b1fa-4c15-9e78-18f34f064d01 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:13.449997 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:13.455616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-d0b9e6b2-b1fa-4c15-9e78-18f34f064d01 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '48cfad6adf9c4e76995345e71d84fb2a', 'request_id': 'req-d0b9e6b2-b1fa-4c15-9e78-18f34f064d01'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.466015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-d0b9e6b2-b1fa-4c15-9e78-18f34f064d01 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '48cfad6adf9c4e76995345e71d84fb2a', 'request_id': 'req-d0b9e6b2-b1fa-4c15-9e78-18f34f064d01'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.472903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-d0b9e6b2-b1fa-4c15-9e78-18f34f064d01 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '48cfad6adf9c4e76995345e71d84fb2a', 'request_id': 'req-d0b9e6b2-b1fa-4c15-9e78-18f34f064d01'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.485783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-d0b9e6b2-b1fa-4c15-9e78-18f34f064d01 admin admin] The token cache is being invalidate because project 48cfad6adf9c4e76995345e71d84fb2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:13.486077 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-d0b9e6b2-b1fa-4c15-9e78-18f34f064d01 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.486772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-d0b9e6b2-b1fa-4c15-9e78-18f34f064d01 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.489069 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1355/2692] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:13 2022] DELETE /identity/v3/projects/48cfad6adf9c4e76995345e71d84fb2a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:13.492415 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3c819-a57b-4f9c-a442-0d725f7f26a4 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:13.493808 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c4a3c819-a57b-4f9c-a442-0d725f7f26a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:13.541563 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:13.590792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3c819-a57b-4f9c-a442-0d725f7f26a4 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:13.667439 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe8a389-36f6-474d-9707-03b93bae56a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:13.668338 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8a389-36f6-474d-9707-03b93bae56a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:13.668626 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8a389-36f6-474d-9707-03b93bae56a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:13.668906 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe8a389-36f6-474d-9707-03b93bae56a8 admin admin] PATH_INFO: `/v3/users/47c4b0d434d34207a36d59d12f0072d2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:13.670120 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8a389-36f6-474d-9707-03b93bae56a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47c4b0d434d34207a36d59d12f0072d2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:13.672249 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe8a389-36f6-474d-9707-03b93bae56a8 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:13.708430 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9e6b2-b1fa-4c15-9e78-18f34f064d01 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:13.710757 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-d0b9e6b2-b1fa-4c15-9e78-18f34f064d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:13.733026 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b9e6b2-b1fa-4c15-9e78-18f34f064d01 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:13.736531 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cbe8a389-36f6-474d-9707-03b93bae56a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47c4b0d434d34207a36d59d12f0072d2'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.737902 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afbe2e37-ed49-43e9-a32b-8a8907936948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:13.738139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cbe8a389-36f6-474d-9707-03b93bae56a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47c4b0d434d34207a36d59d12f0072d2'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.739197 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe2e37-ed49-43e9-a32b-8a8907936948 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:13.739574 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe2e37-ed49-43e9-a32b-8a8907936948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:13.740460 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afbe2e37-ed49-43e9-a32b-8a8907936948 admin admin] PATH_INFO: `/v3/projects/122c5451dae941cca46a2c253b23a2d7` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:13.742659 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe2e37-ed49-43e9-a32b-8a8907936948 admin admin] RBAC: Authorizing `identity:delete_project(project_id=122c5451dae941cca46a2c253b23a2d7)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:13.744776 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afbe2e37-ed49-43e9-a32b-8a8907936948 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:13.747687 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cbe8a389-36f6-474d-9707-03b93bae56a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47c4b0d434d34207a36d59d12f0072d2'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.748476 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cbe8a389-36f6-474d-9707-03b93bae56a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47c4b0d434d34207a36d59d12f0072d2'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.762006 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-cbe8a389-36f6-474d-9707-03b93bae56a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47c4b0d434d34207a36d59d12f0072d2'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.778103 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1338/2693] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:13 2022] DELETE /identity/v3/users/47c4b0d434d34207a36d59d12f0072d2 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:13.798878 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-afbe2e37-ed49-43e9-a32b-8a8907936948 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '122c5451dae941cca46a2c253b23a2d7', 'request_id': 'req-afbe2e37-ed49-43e9-a32b-8a8907936948'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.805913 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-afbe2e37-ed49-43e9-a32b-8a8907936948 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '122c5451dae941cca46a2c253b23a2d7', 'request_id': 'req-afbe2e37-ed49-43e9-a32b-8a8907936948'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.811917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-afbe2e37-ed49-43e9-a32b-8a8907936948 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '122c5451dae941cca46a2c253b23a2d7', 'request_id': 'req-afbe2e37-ed49-43e9-a32b-8a8907936948'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.825717 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-afbe2e37-ed49-43e9-a32b-8a8907936948 admin admin] The token cache is being invalidate because project 122c5451dae941cca46a2c253b23a2d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:13.826012 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-afbe2e37-ed49-43e9-a32b-8a8907936948 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.826745 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-afbe2e37-ed49-43e9-a32b-8a8907936948 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:13.828495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1356/2694] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:13 2022] DELETE /identity/v3/projects/122c5451dae941cca46a2c253b23a2d7 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:13.835835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243278f0-db80-41fc-990d-d50b311f5f79 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:13.836174 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243278f0-db80-41fc-990d-d50b311f5f79 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:13.836560 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243278f0-db80-41fc-990d-d50b311f5f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:13.857026 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-243278f0-db80-41fc-990d-d50b311f5f79 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:13.858538 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-243278f0-db80-41fc-990d-d50b311f5f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:13.920088 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:13.942532 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1339/2695] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:13.956553 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da997e-3cae-46f6-9148-7085e71d2155 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:13.957114 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da997e-3cae-46f6-9148-7085e71d2155 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:13.957396 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2da997e-3cae-46f6-9148-7085e71d2155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:13.981500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-c2da997e-3cae-46f6-9148-7085e71d2155 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:13.982805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c2da997e-3cae-46f6-9148-7085e71d2155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:13.996628 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1357/2696] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:14.000078 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243278f0-db80-41fc-990d-d50b311f5f79 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.009914 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243278f0-db80-41fc-990d-d50b311f5f79 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.014768 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dd7f1-2dbc-4507-a720-04068acda4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.015428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dd7f1-2dbc-4507-a720-04068acda4c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.015719 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dd7f1-2dbc-4507-a720-04068acda4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.016005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510dd7f1-2dbc-4507-a720-04068acda4c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.016731 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510dd7f1-2dbc-4507-a720-04068acda4c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:08:14.017044 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510dd7f1-2dbc-4507-a720-04068acda4c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.018695 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510dd7f1-2dbc-4507-a720-04068acda4c0 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.034431 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1340/2697] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:08:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:08:14.041200 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da997e-3cae-46f6-9148-7085e71d2155 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.048395 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2da997e-3cae-46f6-9148-7085e71d2155 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.053144 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603fdabf-5776-471b-bb99-a525f69d70c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.054382 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fdabf-5776-471b-bb99-a525f69d70c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.054715 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fdabf-5776-471b-bb99-a525f69d70c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.055016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603fdabf-5776-471b-bb99-a525f69d70c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.055736 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603fdabf-5776-471b-bb99-a525f69d70c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.057631 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603fdabf-5776-471b-bb99-a525f69d70c3 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.088188 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dd7f1-2dbc-4507-a720-04068acda4c0 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.090706 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-510dd7f1-2dbc-4507-a720-04068acda4c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:14.090897 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1358/2698] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:14 2022] POST /identity/v3/projects => generated 365 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:08:14.094899 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603fdabf-5776-471b-bb99-a525f69d70c3 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.107128 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510dd7f1-2dbc-4507-a720-04068acda4c0 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.170248 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603fdabf-5776-471b-bb99-a525f69d70c3 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.175076 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ce664-f8ae-4a0d-95a5-367bbabab9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.175755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ce664-f8ae-4a0d-95a5-367bbabab9b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.176037 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ce664-f8ae-4a0d-95a5-367bbabab9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.176313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158ce664-f8ae-4a0d-95a5-367bbabab9b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.177004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ce664-f8ae-4a0d-95a5-367bbabab9b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.179491 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158ce664-f8ae-4a0d-95a5-367bbabab9b2 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.179613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a905974b-7fca-44b4-80be-81378269edf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.180442 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905974b-7fca-44b4-80be-81378269edf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.180728 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905974b-7fca-44b4-80be-81378269edf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.181007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a905974b-7fca-44b4-80be-81378269edf7 admin admin] PATH_INFO: `/v3/projects/04d3c9e94d2f4e83a269b093ac2b318a` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.182192 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905974b-7fca-44b4-80be-81378269edf7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04d3c9e94d2f4e83a269b093ac2b318a)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.184132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a905974b-7fca-44b4-80be-81378269edf7 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.202458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1359/2699] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:14 2022] POST /identity/v3/users => generated 283 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:08:14.206436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ce664-f8ae-4a0d-95a5-367bbabab9b2 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.222296 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158ce664-f8ae-4a0d-95a5-367bbabab9b2 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.228524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395706d-bdb5-4e18-b909-ba667d207b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.229326 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395706d-bdb5-4e18-b909-ba667d207b7f admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.229689 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395706d-bdb5-4e18-b909-ba667d207b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.230100 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395706d-bdb5-4e18-b909-ba667d207b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.230885 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395706d-bdb5-4e18-b909-ba667d207b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.232558 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395706d-bdb5-4e18-b909-ba667d207b7f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.243913 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1360/2700] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:14.247868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395706d-bdb5-4e18-b909-ba667d207b7f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.249001 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-a905974b-7fca-44b4-80be-81378269edf7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04d3c9e94d2f4e83a269b093ac2b318a', 'request_id': 'req-a905974b-7fca-44b4-80be-81378269edf7'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:14.254572 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=83,keystone:DELETE=11,keystone:INSERT=6 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:14.259977 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-a905974b-7fca-44b4-80be-81378269edf7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04d3c9e94d2f4e83a269b093ac2b318a', 'request_id': 'req-a905974b-7fca-44b4-80be-81378269edf7'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:14.260763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395706d-bdb5-4e18-b909-ba667d207b7f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.267722 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-a905974b-7fca-44b4-80be-81378269edf7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04d3c9e94d2f4e83a269b093ac2b318a', 'request_id': 'req-a905974b-7fca-44b4-80be-81378269edf7'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:14.280874 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-a905974b-7fca-44b4-80be-81378269edf7 admin admin] The token cache is being invalidate because project 04d3c9e94d2f4e83a269b093ac2b318a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:14.281257 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-a905974b-7fca-44b4-80be-81378269edf7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:14.282025 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-a905974b-7fca-44b4-80be-81378269edf7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:14.283739 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1341/2701] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:14 2022] DELETE /identity/v3/projects/04d3c9e94d2f4e83a269b093ac2b318a => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:14.288773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5b57d-d59a-42b7-af73-d25d26c50db8 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.289181 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5b57d-d59a-42b7-af73-d25d26c50db8 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.289541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd5b57d-d59a-42b7-af73-d25d26c50db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.308470 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-8fd5b57d-d59a-42b7-af73-d25d26c50db8 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:14.309800 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-8fd5b57d-d59a-42b7-af73-d25d26c50db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:14.330567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a6a99-2cc5-46f1-a812-5226b7d57d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.331387 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a6a99-2cc5-46f1-a812-5226b7d57d93 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.331673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a6a99-2cc5-46f1-a812-5226b7d57d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.331958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3a6a99-2cc5-46f1-a812-5226b7d57d93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.332686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a6a99-2cc5-46f1-a812-5226b7d57d93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.334273 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3a6a99-2cc5-46f1-a812-5226b7d57d93 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.344760 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1361/2702] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:08:14.349009 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a6a99-2cc5-46f1-a812-5226b7d57d93 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.351317 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-de3a6a99-2cc5-46f1-a812-5226b7d57d93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:14.364645 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3a6a99-2cc5-46f1-a812-5226b7d57d93 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.369516 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a34a7f6-b8a3-4a5e-be7f-e4dfde610e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.370485 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34a7f6-b8a3-4a5e-be7f-e4dfde610e60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.370810 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34a7f6-b8a3-4a5e-be7f-e4dfde610e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.371096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a34a7f6-b8a3-4a5e-be7f-e4dfde610e60 admin admin] PATH_INFO: `/v3/projects/a8311e09374b4c64be1ffa3609b888ea/users/f7f68a84de1646e0a647a34e9be17088/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.379972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1342/2703] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:14.386555 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2257f62-cc9b-4e1d-b4de-35ebaf0733ce None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.386873 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2257f62-cc9b-4e1d-b4de-35ebaf0733ce None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.387151 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2257f62-cc9b-4e1d-b4de-35ebaf0733ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.387255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34a7f6-b8a3-4a5e-be7f-e4dfde610e60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8311e09374b4c64be1ffa3609b888ea, user_id=f7f68a84de1646e0a647a34e9be17088, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.389915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a34a7f6-b8a3-4a5e-be7f-e4dfde610e60 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.404743 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1362/2704] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:14 2022] PUT /identity/v3/projects/a8311e09374b4c64be1ffa3609b888ea/users/f7f68a84de1646e0a647a34e9be17088/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:14.411631 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-c2257f62-cc9b-4e1d-b4de-35ebaf0733ce None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:14.413428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c2257f62-cc9b-4e1d-b4de-35ebaf0733ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:14.416944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766355d2-6ba8-4966-9c4a-27e3ffd4e48c None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.417243 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766355d2-6ba8-4966-9c4a-27e3ffd4e48c None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.417523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766355d2-6ba8-4966-9c4a-27e3ffd4e48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.424479 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:14.452918 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-766355d2-6ba8-4966-9c4a-27e3ffd4e48c None None] MFA Rules not processed for user `f7f68a84de1646e0a647a34e9be17088`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:14.454800 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-766355d2-6ba8-4966-9c4a-27e3ffd4e48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:14.482379 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1343/2705] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:14.494485 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2257f62-cc9b-4e1d-b4de-35ebaf0733ce None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.505640 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2257f62-cc9b-4e1d-b4de-35ebaf0733ce None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.512661 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92fee7-2a94-4159-a5fb-a79a645f40e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.513514 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92fee7-2a94-4159-a5fb-a79a645f40e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.513895 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92fee7-2a94-4159-a5fb-a79a645f40e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.514179 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c92fee7-2a94-4159-a5fb-a79a645f40e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.514973 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92fee7-2a94-4159-a5fb-a79a645f40e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:08:14.515283 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92fee7-2a94-4159-a5fb-a79a645f40e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.516882 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c92fee7-2a94-4159-a5fb-a79a645f40e9 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.532792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1344/2706] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:08:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 23:08:14.533627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1363/2707] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:14 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:14.535510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92fee7-2a94-4159-a5fb-a79a645f40e9 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.535671 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766355d2-6ba8-4966-9c4a-27e3ffd4e48c None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.548133 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c92fee7-2a94-4159-a5fb-a79a645f40e9 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.548899 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766355d2-6ba8-4966-9c4a-27e3ffd4e48c None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.553111 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36757c68-957e-4638-9495-7d229074f766 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.554320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36757c68-957e-4638-9495-7d229074f766 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.554625 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36757c68-957e-4638-9495-7d229074f766 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.554901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36757c68-957e-4638-9495-7d229074f766 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.555072 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39d491-4cba-417c-aced-d538dc17e7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.555745 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36757c68-957e-4638-9495-7d229074f766 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.556223 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39d491-4cba-417c-aced-d538dc17e7ff admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.556638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39d491-4cba-417c-aced-d538dc17e7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.557139 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db39d491-4cba-417c-aced-d538dc17e7ff admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.557635 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36757c68-957e-4638-9495-7d229074f766 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.558039 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39d491-4cba-417c-aced-d538dc17e7ff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.560798 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db39d491-4cba-417c-aced-d538dc17e7ff admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.597091 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1345/2708] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:14 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:14.597281 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1364/2709] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:14 2022] POST /identity/v3/projects => generated 367 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:14.597384 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36757c68-957e-4638-9495-7d229074f766 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.598028 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39d491-4cba-417c-aced-d538dc17e7ff admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.660675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:14.674495 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36757c68-957e-4638-9495-7d229074f766 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.680214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db39d491-4cba-417c-aced-d538dc17e7ff admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.680583 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760a06d-3452-48c4-9d00-7aa3c622b8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.681298 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760a06d-3452-48c4-9d00-7aa3c622b8da admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.681611 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760a06d-3452-48c4-9d00-7aa3c622b8da admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.681892 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a760a06d-3452-48c4-9d00-7aa3c622b8da admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.682736 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a760a06d-3452-48c4-9d00-7aa3c622b8da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.685116 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a760a06d-3452-48c4-9d00-7aa3c622b8da admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.685284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e9586-8543-401c-af9f-0c5bbc3132b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.685859 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e9586-8543-401c-af9f-0c5bbc3132b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.686145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e9586-8543-401c-af9f-0c5bbc3132b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.686520 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6e9586-8543-401c-af9f-0c5bbc3132b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.687163 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e9586-8543-401c-af9f-0c5bbc3132b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.689097 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6e9586-8543-401c-af9f-0c5bbc3132b4 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.712644 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1346/2710] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:14 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:08:14.713394 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1365/2711] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:14 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:08:14.713918 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760a06d-3452-48c4-9d00-7aa3c622b8da admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.714453 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e9586-8543-401c-af9f-0c5bbc3132b4 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.725005 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6e9586-8543-401c-af9f-0c5bbc3132b4 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.725215 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a760a06d-3452-48c4-9d00-7aa3c622b8da admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.730356 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3b1c5-88bf-4b39-8b50-ac626eb68243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.730538 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce46278-2bdd-42e3-86f6-d67af21a3ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.731110 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3b1c5-88bf-4b39-8b50-ac626eb68243 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.731224 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce46278-2bdd-42e3-86f6-d67af21a3ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.731397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3b1c5-88bf-4b39-8b50-ac626eb68243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.731519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce46278-2bdd-42e3-86f6-d67af21a3ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.731676 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb3b1c5-88bf-4b39-8b50-ac626eb68243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.731825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce46278-2bdd-42e3-86f6-d67af21a3ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.732394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3b1c5-88bf-4b39-8b50-ac626eb68243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.732524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce46278-2bdd-42e3-86f6-d67af21a3ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.733980 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb3b1c5-88bf-4b39-8b50-ac626eb68243 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.734092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce46278-2bdd-42e3-86f6-d67af21a3ec0 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.744753 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1366/2712] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:14.746778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1347/2713] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:14.750727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce46278-2bdd-42e3-86f6-d67af21a3ec0 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.750936 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3b1c5-88bf-4b39-8b50-ac626eb68243 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.761179 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce46278-2bdd-42e3-86f6-d67af21a3ec0 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.761334 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb3b1c5-88bf-4b39-8b50-ac626eb68243 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.765779 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f59177-1402-4966-903c-b42bc5fb239f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.766191 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1447319-9c8d-4a9e-acd5-3acaa0f42495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.766419 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f59177-1402-4966-903c-b42bc5fb239f admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.766736 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f59177-1402-4966-903c-b42bc5fb239f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.767014 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f59177-1402-4966-903c-b42bc5fb239f admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.767113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1447319-9c8d-4a9e-acd5-3acaa0f42495 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.767378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1447319-9c8d-4a9e-acd5-3acaa0f42495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.767656 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1447319-9c8d-4a9e-acd5-3acaa0f42495 admin admin] PATH_INFO: `/v3/projects/035da623e1a349e78231b4df697c3c1f/users/4245a74baabc4de4b62b007a3b16903f/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.767762 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f59177-1402-4966-903c-b42bc5fb239f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.769313 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f59177-1402-4966-903c-b42bc5fb239f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.782621 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1367/2714] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:14.786708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f59177-1402-4966-903c-b42bc5fb239f admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.790119 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1447319-9c8d-4a9e-acd5-3acaa0f42495 admin admin] RBAC: Authorizing `identity:create_grant(project_id=035da623e1a349e78231b4df697c3c1f, user_id=4245a74baabc4de4b62b007a3b16903f, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.792841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1447319-9c8d-4a9e-acd5-3acaa0f42495 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.797022 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f59177-1402-4966-903c-b42bc5fb239f admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.801924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c38d90-36ee-4754-abe4-9b0998c19e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.802830 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c38d90-36ee-4754-abe4-9b0998c19e6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.803147 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c38d90-36ee-4754-abe4-9b0998c19e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.803426 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c38d90-36ee-4754-abe4-9b0998c19e6f admin admin] PATH_INFO: `/v3/projects/939e1e8098a046b2b4b174c8f3415783/users/e1ca8cd7040d467ab27496d634dff829/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.806720 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1348/2715] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:14 2022] PUT /identity/v3/projects/035da623e1a349e78231b4df697c3c1f/users/4245a74baabc4de4b62b007a3b16903f/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:14.810481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1447319-9c8d-4a9e-acd5-3acaa0f42495 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.821121 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c38d90-36ee-4754-abe4-9b0998c19e6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=939e1e8098a046b2b4b174c8f3415783, user_id=e1ca8cd7040d467ab27496d634dff829, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.823790 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c38d90-36ee-4754-abe4-9b0998c19e6f admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.845018 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1368/2716] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:14 2022] PUT /identity/v3/projects/939e1e8098a046b2b4b174c8f3415783/users/e1ca8cd7040d467ab27496d634dff829/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:14.855985 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2600440-4a7f-4b3b-b03b-363399ef7b2b None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.856283 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2600440-4a7f-4b3b-b03b-363399ef7b2b None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.856567 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2600440-4a7f-4b3b-b03b-363399ef7b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.857196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:14.884861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1447319-9c8d-4a9e-acd5-3acaa0f42495 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.885319 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-c2600440-4a7f-4b3b-b03b-363399ef7b2b None None] MFA Rules not processed for user `e1ca8cd7040d467ab27496d634dff829`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:14.886536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c2600440-4a7f-4b3b-b03b-363399ef7b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:14.941750 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bab326-db2c-4948-9dfb-3cf3468c3bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.942467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bab326-db2c-4948-9dfb-3cf3468c3bdf admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.942787 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bab326-db2c-4948-9dfb-3cf3468c3bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.943073 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bab326-db2c-4948-9dfb-3cf3468c3bdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.943799 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bab326-db2c-4948-9dfb-3cf3468c3bdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.946231 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bab326-db2c-4948-9dfb-3cf3468c3bdf admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.956170 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1369/2717] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:14 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:14.956340 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1349/2718] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:14 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:08:14.960700 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bab326-db2c-4948-9dfb-3cf3468c3bdf admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.960850 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2600440-4a7f-4b3b-b03b-363399ef7b2b None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:14.972836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bab326-db2c-4948-9dfb-3cf3468c3bdf admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.973083 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2600440-4a7f-4b3b-b03b-363399ef7b2b None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:14.975813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11357412-f910-4580-9cc5-394030c53019 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.976718 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11357412-f910-4580-9cc5-394030c53019 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.977014 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11357412-f910-4580-9cc5-394030c53019 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.977446 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11357412-f910-4580-9cc5-394030c53019 admin admin] PATH_INFO: `/v3/projects/035da623e1a349e78231b4df697c3c1f/users/4245a74baabc4de4b62b007a3b16903f/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.977553 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5fa28-f5cf-4165-9b16-9c801f9a5fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:14.978224 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5fa28-f5cf-4165-9b16-9c801f9a5fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:14.978513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5fa28-f5cf-4165-9b16-9c801f9a5fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:14.978839 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e5fa28-f5cf-4165-9b16-9c801f9a5fe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:14.979481 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11357412-f910-4580-9cc5-394030c53019 admin admin] RBAC: Authorizing `identity:create_grant(project_id=035da623e1a349e78231b4df697c3c1f, user_id=4245a74baabc4de4b62b007a3b16903f, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.979587 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5fa28-f5cf-4165-9b16-9c801f9a5fe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:14.981458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e5fa28-f5cf-4165-9b16-9c801f9a5fe7 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.982128 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11357412-f910-4580-9cc5-394030c53019 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:14.993513 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1350/2719] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:14 2022] PUT /identity/v3/projects/035da623e1a349e78231b4df697c3c1f/users/4245a74baabc4de4b62b007a3b16903f/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:15.004577 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14097d-938e-4d73-9d42-3c34c9ada694 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:15.004878 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14097d-938e-4d73-9d42-3c34c9ada694 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:15.005155 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c14097d-938e-4d73-9d42-3c34c9ada694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:15.018772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1370/2720] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:14 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:15.019676 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5fa28-f5cf-4165-9b16-9c801f9a5fe7 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:15.032768 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-0c14097d-938e-4d73-9d42-3c34c9ada694 None None] MFA Rules not processed for user `4245a74baabc4de4b62b007a3b16903f`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:15.033965 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-0c14097d-938e-4d73-9d42-3c34c9ada694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:15.080948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:15.087486 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e5fa28-f5cf-4165-9b16-9c801f9a5fe7 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:15.094959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f50bfe-17d9-4bc0-9e1b-4fa195286a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:15.095641 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f50bfe-17d9-4bc0-9e1b-4fa195286a75 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:15.095933 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f50bfe-17d9-4bc0-9e1b-4fa195286a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:15.096214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f50bfe-17d9-4bc0-9e1b-4fa195286a75 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:15.096905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f50bfe-17d9-4bc0-9e1b-4fa195286a75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:15.098884 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f50bfe-17d9-4bc0-9e1b-4fa195286a75 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:15.110139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1351/2721] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:15 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:15.122790 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1371/2722] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:15 2022] POST /identity/v3/users => generated 297 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:15.123669 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14097d-938e-4d73-9d42-3c34c9ada694 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:15.123934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f50bfe-17d9-4bc0-9e1b-4fa195286a75 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:15.125753 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-0c14097d-938e-4d73-9d42-3c34c9ada694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:15.134672 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f50bfe-17d9-4bc0-9e1b-4fa195286a75 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:15.139441 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d12e13-f210-45d1-9394-e6120be050ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:15.140111 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d12e13-f210-45d1-9394-e6120be050ef admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:15.140402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d12e13-f210-45d1-9394-e6120be050ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:15.140681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d12e13-f210-45d1-9394-e6120be050ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:15.141675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d12e13-f210-45d1-9394-e6120be050ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:15.143051 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d12e13-f210-45d1-9394-e6120be050ef admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:15.157033 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1372/2723] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:15.158046 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d12e13-f210-45d1-9394-e6120be050ef admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:15.168608 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d12e13-f210-45d1-9394-e6120be050ef admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:15.173541 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e48ef-9651-46fe-b5ba-a6c92d57d50c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:15.174441 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e48ef-9651-46fe-b5ba-a6c92d57d50c admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:15.174758 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e48ef-9651-46fe-b5ba-a6c92d57d50c admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:15.175045 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711e48ef-9651-46fe-b5ba-a6c92d57d50c admin admin] PATH_INFO: `/v3/projects/91f70e5c4d924ea68661b9d2835c4166/users/e093141ab5a540f89d5f4b5e6d42de73/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:15.190787 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c14097d-938e-4d73-9d42-3c34c9ada694 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:15.192137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e48ef-9651-46fe-b5ba-a6c92d57d50c admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f70e5c4d924ea68661b9d2835c4166, user_id=e093141ab5a540f89d5f4b5e6d42de73, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:15.195588 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711e48ef-9651-46fe-b5ba-a6c92d57d50c admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:15.196300 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2c362-799a-43a6-bfc9-f25120f72951 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:15.196963 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2c362-799a-43a6-bfc9-f25120f72951 service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:15.197250 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2c362-799a-43a6-bfc9-f25120f72951 service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:15.197527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf2c362-799a-43a6-bfc9-f25120f72951 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:15.204778 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:15.209465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1373/2724] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:15 2022] PUT /identity/v3/projects/91f70e5c4d924ea68661b9d2835c4166/users/e093141ab5a540f89d5f4b5e6d42de73/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:15.213166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e48ef-9651-46fe-b5ba-a6c92d57d50c admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:15.218150 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2c362-799a-43a6-bfc9-f25120f72951 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:15.220197 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf2c362-799a-43a6-bfc9-f25120f72951 service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:15.239328 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1352/2725] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:15 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:08:15.282247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711e48ef-9651-46fe-b5ba-a6c92d57d50c admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:15.286964 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9d4c4-ca4e-4937-96d5-243584a0ae0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:15.287634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9d4c4-ca4e-4937-96d5-243584a0ae0a admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:15.287975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9d4c4-ca4e-4937-96d5-243584a0ae0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:15.288762 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c9d4c4-ca4e-4937-96d5-243584a0ae0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:15.289495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c9d4c4-ca4e-4937-96d5-243584a0ae0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:15.291106 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c9d4c4-ca4e-4937-96d5-243584a0ae0a admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:15.302302 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1374/2726] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:15 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:08:15.311252 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2c362-799a-43a6-bfc9-f25120f72951 service cinder] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:15.322861 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf2c362-799a-43a6-bfc9-f25120f72951 service cinder] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:15.327960 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e0ee8b-9955-4a9d-9481-5506515adc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:15.329638 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0ee8b-9955-4a9d-9481-5506515adc09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:15.330151 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0ee8b-9955-4a9d-9481-5506515adc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:15.330436 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e0ee8b-9955-4a9d-9481-5506515adc09 admin admin] PATH_INFO: `/v3/projects/91f70e5c4d924ea68661b9d2835c4166/users/e093141ab5a540f89d5f4b5e6d42de73/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:15.332722 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0ee8b-9955-4a9d-9481-5506515adc09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=91f70e5c4d924ea68661b9d2835c4166, user_id=e093141ab5a540f89d5f4b5e6d42de73, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:15.336281 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9d4c4-ca4e-4937-96d5-243584a0ae0a admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:15.336600 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e0ee8b-9955-4a9d-9481-5506515adc09 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:15.348205 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1353/2727] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:15 2022] PUT /identity/v3/projects/91f70e5c4d924ea68661b9d2835c4166/users/e093141ab5a540f89d5f4b5e6d42de73/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:15.359850 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df550a3-c9a2-43d5-980b-31a1332c99fe None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:15.360138 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df550a3-c9a2-43d5-980b-31a1332c99fe None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:15.360416 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df550a3-c9a2-43d5-980b-31a1332c99fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:15.389226 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-6df550a3-c9a2-43d5-980b-31a1332c99fe None None] MFA Rules not processed for user `e093141ab5a540f89d5f4b5e6d42de73`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:15.390453 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6df550a3-c9a2-43d5-980b-31a1332c99fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:15.395458 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c9d4c4-ca4e-4937-96d5-243584a0ae0a admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:15.450087 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09d12e2-19bb-4592-b762-7f41a518ccbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:15.450783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d12e2-19bb-4592-b762-7f41a518ccbb service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:15.451113 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d12e2-19bb-4592-b762-7f41a518ccbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:15.451572 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09d12e2-19bb-4592-b762-7f41a518ccbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:15.462002 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1354/2728] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:15 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:15.473532 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df550a3-c9a2-43d5-980b-31a1332c99fe None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:15.473853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:15.476080 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6df550a3-c9a2-43d5-980b-31a1332c99fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:15.489785 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df550a3-c9a2-43d5-980b-31a1332c99fe None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:15.493433 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e6a66-56d8-46c7-84ec-f63149560f8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:15.494074 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e6a66-56d8-46c7-84ec-f63149560f8a service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:15.494357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e6a66-56d8-46c7-84ec-f63149560f8a service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:15.494663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450e6a66-56d8-46c7-84ec-f63149560f8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:15.503906 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e6a66-56d8-46c7-84ec-f63149560f8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:15.505091 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450e6a66-56d8-46c7-84ec-f63149560f8a service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:15.515046 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1355/2729] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:15 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:08:15.520058 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d12e2-19bb-4592-b762-7f41a518ccbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:15.521436 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09d12e2-19bb-4592-b762-7f41a518ccbb service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:15.536747 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1375/2730] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:15 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:08:15.991649 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e6a66-56d8-46c7-84ec-f63149560f8a service cinder] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:16.052613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450e6a66-56d8-46c7-84ec-f63149560f8a service cinder] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:16.057781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719782b-e894-47cf-a708-183d7a368592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:16.058762 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719782b-e894-47cf-a708-183d7a368592 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:16.059038 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719782b-e894-47cf-a708-183d7a368592 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:16.059326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b719782b-e894-47cf-a708-183d7a368592 admin admin] PATH_INFO: `/v3/users/e1ca8cd7040d467ab27496d634dff829` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:16.060517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719782b-e894-47cf-a708-183d7a368592 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1ca8cd7040d467ab27496d634dff829)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:16.062430 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b719782b-e894-47cf-a708-183d7a368592 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:16.113152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b719782b-e894-47cf-a708-183d7a368592 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1ca8cd7040d467ab27496d634dff829'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:16.113807 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b719782b-e894-47cf-a708-183d7a368592 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1ca8cd7040d467ab27496d634dff829'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:16.119618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=92,keystone:INSERT=1,keystone:DELETE=7 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:16.121752 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b719782b-e894-47cf-a708-183d7a368592 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1ca8cd7040d467ab27496d634dff829'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:16.122696 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b719782b-e894-47cf-a708-183d7a368592 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1ca8cd7040d467ab27496d634dff829'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:16.136836 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b719782b-e894-47cf-a708-183d7a368592 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1ca8cd7040d467ab27496d634dff829'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:16.148875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1356/2731] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:15 2022] DELETE /identity/v3/users/e1ca8cd7040d467ab27496d634dff829 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:16.153753 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f2999-bf70-4da9-bf68-b5fbfdffa17c None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:16.154239 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f2999-bf70-4da9-bf68-b5fbfdffa17c None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:16.154338 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513f2999-bf70-4da9-bf68-b5fbfdffa17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:16.171229 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-513f2999-bf70-4da9-bf68-b5fbfdffa17c None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:16.172438 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-513f2999-bf70-4da9-bf68-b5fbfdffa17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:16.233368 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1376/2732] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:16.243789 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719782b-e894-47cf-a708-183d7a368592 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:16.245415 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-b719782b-e894-47cf-a708-183d7a368592 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:16.309792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b719782b-e894-47cf-a708-183d7a368592 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:16.313707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2d41f-76ca-4615-b475-8f378e759736 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:16.314385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2d41f-76ca-4615-b475-8f378e759736 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:16.314765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2d41f-76ca-4615-b475-8f378e759736 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:16.315118 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2d41f-76ca-4615-b475-8f378e759736 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:16.324520 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2d41f-76ca-4615-b475-8f378e759736 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:16.325956 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2d41f-76ca-4615-b475-8f378e759736 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:16.335415 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1357/2733] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:16.600235 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f2999-bf70-4da9-bf68-b5fbfdffa17c None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:16.602289 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-513f2999-bf70-4da9-bf68-b5fbfdffa17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:16.614278 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513f2999-bf70-4da9-bf68-b5fbfdffa17c None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:16.618960 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4dd505-0aa0-4ec3-8c84-22f33cfb8c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:16.619836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4dd505-0aa0-4ec3-8c84-22f33cfb8c88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:16.620128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4dd505-0aa0-4ec3-8c84-22f33cfb8c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:16.620408 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4dd505-0aa0-4ec3-8c84-22f33cfb8c88 admin admin] PATH_INFO: `/v3/projects/939e1e8098a046b2b4b174c8f3415783` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:16.621602 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4dd505-0aa0-4ec3-8c84-22f33cfb8c88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=939e1e8098a046b2b4b174c8f3415783)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:16.623678 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4dd505-0aa0-4ec3-8c84-22f33cfb8c88 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:16.673016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6a4dd505-0aa0-4ec3-8c84-22f33cfb8c88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '939e1e8098a046b2b4b174c8f3415783', 'request_id': 'req-6a4dd505-0aa0-4ec3-8c84-22f33cfb8c88'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:16.679312 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6a4dd505-0aa0-4ec3-8c84-22f33cfb8c88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '939e1e8098a046b2b4b174c8f3415783', 'request_id': 'req-6a4dd505-0aa0-4ec3-8c84-22f33cfb8c88'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:16.685144 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6a4dd505-0aa0-4ec3-8c84-22f33cfb8c88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '939e1e8098a046b2b4b174c8f3415783', 'request_id': 'req-6a4dd505-0aa0-4ec3-8c84-22f33cfb8c88'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:16.699032 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6a4dd505-0aa0-4ec3-8c84-22f33cfb8c88 admin admin] The token cache is being invalidate because project 939e1e8098a046b2b4b174c8f3415783 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:16.699337 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6a4dd505-0aa0-4ec3-8c84-22f33cfb8c88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:16.699939 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-6a4dd505-0aa0-4ec3-8c84-22f33cfb8c88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:16.701495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1377/2734] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:16 2022] DELETE /identity/v3/projects/939e1e8098a046b2b4b174c8f3415783 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:16.705397 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2d41f-76ca-4615-b475-8f378e759736 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:16.715792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2d41f-76ca-4615-b475-8f378e759736 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:16.774047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0e4ffb-e238-4509-8b5e-8730235ffff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:16.774976 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e4ffb-e238-4509-8b5e-8730235ffff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:16.775268 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e4ffb-e238-4509-8b5e-8730235ffff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:16.775552 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0e4ffb-e238-4509-8b5e-8730235ffff6 admin admin] PATH_INFO: `/v3/users/e093141ab5a540f89d5f4b5e6d42de73` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:16.776794 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e4ffb-e238-4509-8b5e-8730235ffff6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e093141ab5a540f89d5f4b5e6d42de73)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:16.778756 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0e4ffb-e238-4509-8b5e-8730235ffff6 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:16.824016 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5a0e4ffb-e238-4509-8b5e-8730235ffff6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e093141ab5a540f89d5f4b5e6d42de73'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:16.824665 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5a0e4ffb-e238-4509-8b5e-8730235ffff6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e093141ab5a540f89d5f4b5e6d42de73'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:16.831599 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5a0e4ffb-e238-4509-8b5e-8730235ffff6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e093141ab5a540f89d5f4b5e6d42de73'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:16.832219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5a0e4ffb-e238-4509-8b5e-8730235ffff6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e093141ab5a540f89d5f4b5e6d42de73'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:16.843988 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5a0e4ffb-e238-4509-8b5e-8730235ffff6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e093141ab5a540f89d5f4b5e6d42de73'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:16.846391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:16.858805 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1358/2735] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:16 2022] DELETE /identity/v3/users/e093141ab5a540f89d5f4b5e6d42de73 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:17.418472 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4dd505-0aa0-4ec3-8c84-22f33cfb8c88 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:17.419945 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-6a4dd505-0aa0-4ec3-8c84-22f33cfb8c88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:17.433219 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4dd505-0aa0-4ec3-8c84-22f33cfb8c88 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:17.472910 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:17.495349 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07229319-d4d6-4dea-afb4-d93af578a08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:17.496263 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07229319-d4d6-4dea-afb4-d93af578a08d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:17.496551 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07229319-d4d6-4dea-afb4-d93af578a08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:17.496831 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07229319-d4d6-4dea-afb4-d93af578a08d admin admin] PATH_INFO: `/v3/projects/91f70e5c4d924ea68661b9d2835c4166` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:17.498032 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07229319-d4d6-4dea-afb4-d93af578a08d admin admin] RBAC: Authorizing `identity:delete_project(project_id=91f70e5c4d924ea68661b9d2835c4166)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:17.500011 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07229319-d4d6-4dea-afb4-d93af578a08d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:17.547822 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-07229319-d4d6-4dea-afb4-d93af578a08d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '91f70e5c4d924ea68661b9d2835c4166', 'request_id': 'req-07229319-d4d6-4dea-afb4-d93af578a08d'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:17.553958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-07229319-d4d6-4dea-afb4-d93af578a08d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '91f70e5c4d924ea68661b9d2835c4166', 'request_id': 'req-07229319-d4d6-4dea-afb4-d93af578a08d'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:17.559762 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-07229319-d4d6-4dea-afb4-d93af578a08d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '91f70e5c4d924ea68661b9d2835c4166', 'request_id': 'req-07229319-d4d6-4dea-afb4-d93af578a08d'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:17.572523 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-07229319-d4d6-4dea-afb4-d93af578a08d admin admin] The token cache is being invalidate because project 91f70e5c4d924ea68661b9d2835c4166 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:17.572842 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-07229319-d4d6-4dea-afb4-d93af578a08d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:17.573467 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-07229319-d4d6-4dea-afb4-d93af578a08d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:17.575344 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1378/2736] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:17 2022] DELETE /identity/v3/projects/91f70e5c4d924ea68661b9d2835c4166 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:17.580066 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e24dbe-e834-4f98-990d-a24d75d5aff8 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:17.580589 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e24dbe-e834-4f98-990d-a24d75d5aff8 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:17.580711 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e24dbe-e834-4f98-990d-a24d75d5aff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:17.598263 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-30e24dbe-e834-4f98-990d-a24d75d5aff8 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:17.599504 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-30e24dbe-e834-4f98-990d-a24d75d5aff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:17.661502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1359/2737] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:17.669401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892231a7-1d0c-4af2-b767-5cff0dd872b2 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:17.669700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892231a7-1d0c-4af2-b767-5cff0dd872b2 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:17.669985 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892231a7-1d0c-4af2-b767-5cff0dd872b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:17.687924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-892231a7-1d0c-4af2-b767-5cff0dd872b2 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:17.689167 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-892231a7-1d0c-4af2-b767-5cff0dd872b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:17.697668 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1379/2738] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:17.701865 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e24dbe-e834-4f98-990d-a24d75d5aff8 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:17.712143 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e24dbe-e834-4f98-990d-a24d75d5aff8 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:17.716936 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77d799-f636-477a-9c2c-5b4e91f6ac78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:17.717589 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77d799-f636-477a-9c2c-5b4e91f6ac78 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:17.717984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77d799-f636-477a-9c2c-5b4e91f6ac78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:17.718138 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af77d799-f636-477a-9c2c-5b4e91f6ac78 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:17.718941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77d799-f636-477a-9c2c-5b4e91f6ac78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:08:17.719196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77d799-f636-477a-9c2c-5b4e91f6ac78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:17.720770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af77d799-f636-477a-9c2c-5b4e91f6ac78 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:17.735683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1360/2739] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:08:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:08:17.739990 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892231a7-1d0c-4af2-b767-5cff0dd872b2 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:17.750177 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892231a7-1d0c-4af2-b767-5cff0dd872b2 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:17.754501 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47e11b-58b4-4560-85cd-20e03bd43436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:17.755258 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47e11b-58b4-4560-85cd-20e03bd43436 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:17.755500 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47e11b-58b4-4560-85cd-20e03bd43436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:17.755930 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de47e11b-58b4-4560-85cd-20e03bd43436 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:17.757364 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de47e11b-58b4-4560-85cd-20e03bd43436 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:17.759797 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de47e11b-58b4-4560-85cd-20e03bd43436 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:17.789768 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1380/2740] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:17 2022] POST /identity/v3/projects => generated 361 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:08:17.793622 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77d799-f636-477a-9c2c-5b4e91f6ac78 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:17.803458 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af77d799-f636-477a-9c2c-5b4e91f6ac78 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:17.864445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23206cc-39cc-451e-ae66-dd4485e460da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:17.865132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23206cc-39cc-451e-ae66-dd4485e460da admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:17.865428 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23206cc-39cc-451e-ae66-dd4485e460da admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:17.865714 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23206cc-39cc-451e-ae66-dd4485e460da admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:17.866447 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23206cc-39cc-451e-ae66-dd4485e460da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:17.868408 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23206cc-39cc-451e-ae66-dd4485e460da admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:17.888798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1361/2741] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:17 2022] POST /identity/v3/users => generated 281 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:08:17.892898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47e11b-58b4-4560-85cd-20e03bd43436 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:17.905163 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de47e11b-58b4-4560-85cd-20e03bd43436 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:17.909805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0745ae2-5848-41de-84aa-0a431a64a891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:17.910473 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0745ae2-5848-41de-84aa-0a431a64a891 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:17.910792 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0745ae2-5848-41de-84aa-0a431a64a891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:17.911073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0745ae2-5848-41de-84aa-0a431a64a891 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:17.911851 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0745ae2-5848-41de-84aa-0a431a64a891 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:17.913391 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0745ae2-5848-41de-84aa-0a431a64a891 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:17.924015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1381/2742] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:17.927985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23206cc-39cc-451e-ae66-dd4485e460da admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:17.937903 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23206cc-39cc-451e-ae66-dd4485e460da admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:17.942527 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569d8498-050a-46ef-818d-2858804f9f63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:17.943292 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d8498-050a-46ef-818d-2858804f9f63 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:17.943571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d8498-050a-46ef-818d-2858804f9f63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:17.944352 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569d8498-050a-46ef-818d-2858804f9f63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:17.945091 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d8498-050a-46ef-818d-2858804f9f63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:17.947054 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569d8498-050a-46ef-818d-2858804f9f63 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:17.957612 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1362/2743] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:17.962759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0745ae2-5848-41de-84aa-0a431a64a891 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:17.973488 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0745ae2-5848-41de-84aa-0a431a64a891 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:17.978717 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1cafb7-6889-4a00-9075-15f816583bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:17.980000 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cafb7-6889-4a00-9075-15f816583bfb admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:17.980533 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cafb7-6889-4a00-9075-15f816583bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:17.980820 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1cafb7-6889-4a00-9075-15f816583bfb admin admin] PATH_INFO: `/v3/projects/4ad4489c7895419a87a70a67055cbca7/users/7a31edb84ade4fc18636fef2bacf7c00/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:17.998442 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1cafb7-6889-4a00-9075-15f816583bfb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ad4489c7895419a87a70a67055cbca7, user_id=7a31edb84ade4fc18636fef2bacf7c00, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:18.001085 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1cafb7-6889-4a00-9075-15f816583bfb admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:18.013662 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1382/2744] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:17 2022] PUT /identity/v3/projects/4ad4489c7895419a87a70a67055cbca7/users/7a31edb84ade4fc18636fef2bacf7c00/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:18.024878 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f00ddb-c6e4-4d1c-a9b6-b0e49d74c347 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:18.025271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f00ddb-c6e4-4d1c-a9b6-b0e49d74c347 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:18.025624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f00ddb-c6e4-4d1c-a9b6-b0e49d74c347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:18.051697 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-25f00ddb-c6e4-4d1c-a9b6-b0e49d74c347 None None] MFA Rules not processed for user `7a31edb84ade4fc18636fef2bacf7c00`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:18.052937 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-25f00ddb-c6e4-4d1c-a9b6-b0e49d74c347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:18.100894 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:18.120261 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1363/2745] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:18 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:18.150717 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1cafb7-6889-4a00-9075-15f816583bfb admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:18.152402 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ef1cafb7-6889-4a00-9075-15f816583bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:18.211913 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1cafb7-6889-4a00-9075-15f816583bfb admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:18.215764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f9fd5-8d37-4501-a514-59e1440d4a1b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:18.216418 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f9fd5-8d37-4501-a514-59e1440d4a1b service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:18.216828 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f9fd5-8d37-4501-a514-59e1440d4a1b service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:18.216967 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f9fd5-8d37-4501-a514-59e1440d4a1b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:18.226255 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f9fd5-8d37-4501-a514-59e1440d4a1b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:18.227709 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f9fd5-8d37-4501-a514-59e1440d4a1b service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:18.236528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1383/2746] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:18 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:18.257224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f00ddb-c6e4-4d1c-a9b6-b0e49d74c347 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:18.329835 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f00ddb-c6e4-4d1c-a9b6-b0e49d74c347 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:18.335074 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d22cc-a11a-4fc0-9f47-14de3d07fe5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:18.335998 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d22cc-a11a-4fc0-9f47-14de3d07fe5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:18.336292 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d22cc-a11a-4fc0-9f47-14de3d07fe5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:18.336581 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d22cc-a11a-4fc0-9f47-14de3d07fe5b admin admin] PATH_INFO: `/v3/users/7a31edb84ade4fc18636fef2bacf7c00` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:18.337797 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d22cc-a11a-4fc0-9f47-14de3d07fe5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a31edb84ade4fc18636fef2bacf7c00)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:18.343360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d22cc-a11a-4fc0-9f47-14de3d07fe5b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:18.389602 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-3e4d22cc-a11a-4fc0-9f47-14de3d07fe5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a31edb84ade4fc18636fef2bacf7c00'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:18.390904 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-3e4d22cc-a11a-4fc0-9f47-14de3d07fe5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a31edb84ade4fc18636fef2bacf7c00'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:18.400525 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-3e4d22cc-a11a-4fc0-9f47-14de3d07fe5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a31edb84ade4fc18636fef2bacf7c00'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:18.401258 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-3e4d22cc-a11a-4fc0-9f47-14de3d07fe5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a31edb84ade4fc18636fef2bacf7c00'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:18.413633 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-3e4d22cc-a11a-4fc0-9f47-14de3d07fe5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a31edb84ade4fc18636fef2bacf7c00'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:18.426293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1364/2747] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:18 2022] DELETE /identity/v3/users/7a31edb84ade4fc18636fef2bacf7c00 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:18.431342 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fe9f2-5b25-4e31-8901-0d562777aa62 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:18.431800 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fe9f2-5b25-4e31-8901-0d562777aa62 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:18.431951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00fe9f2-5b25-4e31-8901-0d562777aa62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:18.449708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-d00fe9f2-5b25-4e31-8901-0d562777aa62 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:18.450963 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-d00fe9f2-5b25-4e31-8901-0d562777aa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:18.455154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:18.520668 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1384/2748] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:18.531064 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d22cc-a11a-4fc0-9f47-14de3d07fe5b admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:18.536362 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-3e4d22cc-a11a-4fc0-9f47-14de3d07fe5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:18.601348 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d22cc-a11a-4fc0-9f47-14de3d07fe5b admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:18.605798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ecc67f-2d35-4c41-bc62-9e5d54797bfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:18.606687 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecc67f-2d35-4c41-bc62-9e5d54797bfc service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:18.606979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecc67f-2d35-4c41-bc62-9e5d54797bfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:18.607261 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ecc67f-2d35-4c41-bc62-9e5d54797bfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:18.622858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ecc67f-2d35-4c41-bc62-9e5d54797bfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:18.624311 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ecc67f-2d35-4c41-bc62-9e5d54797bfc service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:18.634480 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1365/2749] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:18.952462 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fe9f2-5b25-4e31-8901-0d562777aa62 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:18.954539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-d00fe9f2-5b25-4e31-8901-0d562777aa62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:18.976025 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00fe9f2-5b25-4e31-8901-0d562777aa62 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:18.982996 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac61cb1-0216-4a3c-8342-b4cb3b975e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:18.984781 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac61cb1-0216-4a3c-8342-b4cb3b975e4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:18.985132 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac61cb1-0216-4a3c-8342-b4cb3b975e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:18.985478 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac61cb1-0216-4a3c-8342-b4cb3b975e4d admin admin] PATH_INFO: `/v3/projects/4ad4489c7895419a87a70a67055cbca7` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:18.986828 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac61cb1-0216-4a3c-8342-b4cb3b975e4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ad4489c7895419a87a70a67055cbca7)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:18.988779 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac61cb1-0216-4a3c-8342-b4cb3b975e4d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:19.041330 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-2ac61cb1-0216-4a3c-8342-b4cb3b975e4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ad4489c7895419a87a70a67055cbca7', 'request_id': 'req-2ac61cb1-0216-4a3c-8342-b4cb3b975e4d'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:19.047815 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-2ac61cb1-0216-4a3c-8342-b4cb3b975e4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ad4489c7895419a87a70a67055cbca7', 'request_id': 'req-2ac61cb1-0216-4a3c-8342-b4cb3b975e4d'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:19.054413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-2ac61cb1-0216-4a3c-8342-b4cb3b975e4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ad4489c7895419a87a70a67055cbca7', 'request_id': 'req-2ac61cb1-0216-4a3c-8342-b4cb3b975e4d'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:19.066733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-2ac61cb1-0216-4a3c-8342-b4cb3b975e4d admin admin] The token cache is being invalidate because project 4ad4489c7895419a87a70a67055cbca7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:19.067170 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-2ac61cb1-0216-4a3c-8342-b4cb3b975e4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:19.067964 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-2ac61cb1-0216-4a3c-8342-b4cb3b975e4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:19.069964 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1385/2750] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:18 2022] DELETE /identity/v3/projects/4ad4489c7895419a87a70a67055cbca7 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:19.075255 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498bfb7-2c1b-42c5-b57a-d3d3f853093e None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:19.075549 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498bfb7-2c1b-42c5-b57a-d3d3f853093e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:19.075828 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e498bfb7-2c1b-42c5-b57a-d3d3f853093e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:19.094378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-e498bfb7-2c1b-42c5-b57a-d3d3f853093e None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:19.095601 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-e498bfb7-2c1b-42c5-b57a-d3d3f853093e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:19.102236 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:19.167885 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1366/2751] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:19.179396 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c490918-b450-41a6-97d7-a886a6d2a453 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:19.179816 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c490918-b450-41a6-97d7-a886a6d2a453 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:19.180025 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c490918-b450-41a6-97d7-a886a6d2a453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:19.198897 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-5c490918-b450-41a6-97d7-a886a6d2a453 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:19.200097 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5c490918-b450-41a6-97d7-a886a6d2a453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:19.208575 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1386/2752] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:19.213344 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498bfb7-2c1b-42c5-b57a-d3d3f853093e None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:19.229754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e498bfb7-2c1b-42c5-b57a-d3d3f853093e None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:19.232516 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1d7cc-4bbf-4b6b-9072-6276e1ff79d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:19.233206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1d7cc-4bbf-4b6b-9072-6276e1ff79d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:19.233495 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1d7cc-4bbf-4b6b-9072-6276e1ff79d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:19.233773 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d1d7cc-4bbf-4b6b-9072-6276e1ff79d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:19.234508 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1d7cc-4bbf-4b6b-9072-6276e1ff79d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80088) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:08:19.234862 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1d7cc-4bbf-4b6b-9072-6276e1ff79d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:19.236957 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08d1d7cc-4bbf-4b6b-9072-6276e1ff79d1 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:19.248543 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1367/2753] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:08:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:08:19.252860 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c490918-b450-41a6-97d7-a886a6d2a453 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:19.263016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c490918-b450-41a6-97d7-a886a6d2a453 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:19.267709 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6bd44-d51c-4c26-8930-8ad441187547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:19.268364 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6bd44-d51c-4c26-8930-8ad441187547 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:19.268653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6bd44-d51c-4c26-8930-8ad441187547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:19.268933 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6bd44-d51c-4c26-8930-8ad441187547 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:19.269617 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f6bd44-d51c-4c26-8930-8ad441187547 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:19.271546 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f6bd44-d51c-4c26-8930-8ad441187547 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:19.301358 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1387/2754] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:19 2022] POST /identity/v3/projects => generated 375 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:19.302536 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1d7cc-4bbf-4b6b-9072-6276e1ff79d1 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:19.312670 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d1d7cc-4bbf-4b6b-9072-6276e1ff79d1 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:19.371378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a28832-c907-4d0d-b5af-6d69b29e82ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:19.372066 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a28832-c907-4d0d-b5af-6d69b29e82ea admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:19.372349 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a28832-c907-4d0d-b5af-6d69b29e82ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:19.372624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a28832-c907-4d0d-b5af-6d69b29e82ea admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:19.373297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a28832-c907-4d0d-b5af-6d69b29e82ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:19.375279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a28832-c907-4d0d-b5af-6d69b29e82ea admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:19.395219 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1368/2755] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:19 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:08:19.400044 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6bd44-d51c-4c26-8930-8ad441187547 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:19.410513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6bd44-d51c-4c26-8930-8ad441187547 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:19.415611 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc1c35-7d93-49d6-939f-0201d2011a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:19.416280 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc1c35-7d93-49d6-939f-0201d2011a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:19.416576 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc1c35-7d93-49d6-939f-0201d2011a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:19.416874 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cc1c35-7d93-49d6-939f-0201d2011a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:19.417613 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc1c35-7d93-49d6-939f-0201d2011a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:19.419292 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cc1c35-7d93-49d6-939f-0201d2011a41 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:19.430140 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1388/2756] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:19.434232 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a28832-c907-4d0d-b5af-6d69b29e82ea admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:19.444840 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a28832-c907-4d0d-b5af-6d69b29e82ea admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:19.449663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a0bd7c-448d-410a-ae7a-5db8c1d9e087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:19.450315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0bd7c-448d-410a-ae7a-5db8c1d9e087 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:19.450626 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0bd7c-448d-410a-ae7a-5db8c1d9e087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:19.454465 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a0bd7c-448d-410a-ae7a-5db8c1d9e087 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:19.454465 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a0bd7c-448d-410a-ae7a-5db8c1d9e087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:19.454465 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a0bd7c-448d-410a-ae7a-5db8c1d9e087 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:19.467904 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1369/2757] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:19.469127 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc1c35-7d93-49d6-939f-0201d2011a41 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:19.479986 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cc1c35-7d93-49d6-939f-0201d2011a41 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:19.484513 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cf4ed-c65e-4006-a71e-20518a1d7e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:19.485384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cf4ed-c65e-4006-a71e-20518a1d7e6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:19.485671 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cf4ed-c65e-4006-a71e-20518a1d7e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:19.485943 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2cf4ed-c65e-4006-a71e-20518a1d7e6a admin admin] PATH_INFO: `/v3/projects/2a78e95dc9fe48e3a421d3e0d7138a9e/users/b23c891e97bd459a96502be15ae24396/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:19.501086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cf4ed-c65e-4006-a71e-20518a1d7e6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a78e95dc9fe48e3a421d3e0d7138a9e, user_id=b23c891e97bd459a96502be15ae24396, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:19.503777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2cf4ed-c65e-4006-a71e-20518a1d7e6a admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:19.516975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1389/2758] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:19 2022] PUT /identity/v3/projects/2a78e95dc9fe48e3a421d3e0d7138a9e/users/b23c891e97bd459a96502be15ae24396/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:19.527641 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4818e01-35e8-4d81-825e-1cf5453fc65f None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:19.527944 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4818e01-35e8-4d81-825e-1cf5453fc65f None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:19.528224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4818e01-35e8-4d81-825e-1cf5453fc65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:19.554403 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-c4818e01-35e8-4d81-825e-1cf5453fc65f None None] MFA Rules not processed for user `b23c891e97bd459a96502be15ae24396`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:19.555652 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c4818e01-35e8-4d81-825e-1cf5453fc65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:19.623559 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1370/2759] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:19 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:19.627932 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cf4ed-c65e-4006-a71e-20518a1d7e6a admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:19.661213 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:19.694908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2cf4ed-c65e-4006-a71e-20518a1d7e6a admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:19.699901 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626983b1-aa3b-4bae-9ff4-6560a8a05a58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:19.700575 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626983b1-aa3b-4bae-9ff4-6560a8a05a58 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:19.700869 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626983b1-aa3b-4bae-9ff4-6560a8a05a58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:19.701151 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626983b1-aa3b-4bae-9ff4-6560a8a05a58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:19.701855 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626983b1-aa3b-4bae-9ff4-6560a8a05a58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:19.703794 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626983b1-aa3b-4bae-9ff4-6560a8a05a58 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:19.729115 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1390/2760] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:19 2022] POST /identity/v3/projects => generated 377 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:19.733705 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4818e01-35e8-4d81-825e-1cf5453fc65f None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:19.745269 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4818e01-35e8-4d81-825e-1cf5453fc65f None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:19.748908 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:19.806005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bde83-485f-4d65-90c3-0633000d16d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:19.806725 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bde83-485f-4d65-90c3-0633000d16d1 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:19.807014 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bde83-485f-4d65-90c3-0633000d16d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:19.807292 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35bde83-485f-4d65-90c3-0633000d16d1 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:19.807990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35bde83-485f-4d65-90c3-0633000d16d1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:19.809927 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35bde83-485f-4d65-90c3-0633000d16d1 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:19.833526 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1371/2761] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:19 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:08:19.837479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626983b1-aa3b-4bae-9ff4-6560a8a05a58 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:19.849328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626983b1-aa3b-4bae-9ff4-6560a8a05a58 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:19.854294 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da844a-39d6-4949-beef-c68b88c6c088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:19.855000 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da844a-39d6-4949-beef-c68b88c6c088 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:19.855284 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da844a-39d6-4949-beef-c68b88c6c088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:19.855562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8da844a-39d6-4949-beef-c68b88c6c088 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:19.856293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da844a-39d6-4949-beef-c68b88c6c088 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:19.858016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8da844a-39d6-4949-beef-c68b88c6c088 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:19.870397 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1391/2762] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:19.875264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bde83-485f-4d65-90c3-0633000d16d1 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:19.886241 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35bde83-485f-4d65-90c3-0633000d16d1 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:19.891020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7fe2fa-ad23-4f9e-b94b-63e96aae0636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:19.891683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fe2fa-ad23-4f9e-b94b-63e96aae0636 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:19.891974 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fe2fa-ad23-4f9e-b94b-63e96aae0636 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:19.892249 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7fe2fa-ad23-4f9e-b94b-63e96aae0636 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:19.892969 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fe2fa-ad23-4f9e-b94b-63e96aae0636 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:19.894544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7fe2fa-ad23-4f9e-b94b-63e96aae0636 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:19.904622 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1372/2763] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:19.908859 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da844a-39d6-4949-beef-c68b88c6c088 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:19.925201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8da844a-39d6-4949-beef-c68b88c6c088 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:19.930374 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b108e-b8dc-47bd-8b1c-6a8da98d52fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:19.931306 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b108e-b8dc-47bd-8b1c-6a8da98d52fa admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:19.931587 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b108e-b8dc-47bd-8b1c-6a8da98d52fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:19.931860 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3b108e-b8dc-47bd-8b1c-6a8da98d52fa admin admin] PATH_INFO: `/v3/projects/693f0e1850234fe69e386e6b18155a1e/users/1b14014a3885495089c46346211ffe20/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:19.947297 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b108e-b8dc-47bd-8b1c-6a8da98d52fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=693f0e1850234fe69e386e6b18155a1e, user_id=1b14014a3885495089c46346211ffe20, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:19.949903 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3b108e-b8dc-47bd-8b1c-6a8da98d52fa admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:19.963730 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1392/2764] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:19 2022] PUT /identity/v3/projects/693f0e1850234fe69e386e6b18155a1e/users/1b14014a3885495089c46346211ffe20/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:19.975536 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5991a-a5ee-4c51-aaee-d65f9db9e0c4 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:19.975831 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5991a-a5ee-4c51-aaee-d65f9db9e0c4 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:19.976119 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5991a-a5ee-4c51-aaee-d65f9db9e0c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:20.003566 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-a6b5991a-a5ee-4c51-aaee-d65f9db9e0c4 None None] MFA Rules not processed for user `1b14014a3885495089c46346211ffe20`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:20.004732 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-a6b5991a-a5ee-4c51-aaee-d65f9db9e0c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:20.069761 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1373/2765] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:19 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:20.074479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b108e-b8dc-47bd-8b1c-6a8da98d52fa admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:20.138604 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3b108e-b8dc-47bd-8b1c-6a8da98d52fa admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:20.143451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ec8d2-1df8-4f6a-8270-14e1133cffd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:20.144122 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ec8d2-1df8-4f6a-8270-14e1133cffd6 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:20.144411 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ec8d2-1df8-4f6a-8270-14e1133cffd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:20.144695 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ec8d2-1df8-4f6a-8270-14e1133cffd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:20.145387 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ec8d2-1df8-4f6a-8270-14e1133cffd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:20.147363 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ec8d2-1df8-4f6a-8270-14e1133cffd6 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:20.176970 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1393/2766] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:20 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:08:20.181587 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5991a-a5ee-4c51-aaee-d65f9db9e0c4 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:20.192701 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5991a-a5ee-4c51-aaee-d65f9db9e0c4 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:20.251253 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4247c8f-c3cd-4dee-92e7-ed69568fa7d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:20.252008 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4247c8f-c3cd-4dee-92e7-ed69568fa7d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:20.252301 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4247c8f-c3cd-4dee-92e7-ed69568fa7d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:20.252584 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4247c8f-c3cd-4dee-92e7-ed69568fa7d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:20.253281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4247c8f-c3cd-4dee-92e7-ed69568fa7d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:20.255394 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4247c8f-c3cd-4dee-92e7-ed69568fa7d9 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:20.277891 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1374/2767] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:20 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:08:20.283765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ec8d2-1df8-4f6a-8270-14e1133cffd6 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:20.294733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ec8d2-1df8-4f6a-8270-14e1133cffd6 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:20.301131 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5233b-f246-41ef-90f3-4b3af816c1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:20.301826 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5233b-f246-41ef-90f3-4b3af816c1d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:20.302100 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5233b-f246-41ef-90f3-4b3af816c1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:20.302384 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc5233b-f246-41ef-90f3-4b3af816c1d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:20.303733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5233b-f246-41ef-90f3-4b3af816c1d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:20.305347 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc5233b-f246-41ef-90f3-4b3af816c1d4 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:20.320029 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1394/2768] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:20.321692 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4247c8f-c3cd-4dee-92e7-ed69568fa7d9 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:20.328915 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:20.335852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4247c8f-c3cd-4dee-92e7-ed69568fa7d9 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:20.340794 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3a078-8d10-4e3a-bc0b-427134e30205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:20.341660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3a078-8d10-4e3a-bc0b-427134e30205 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:20.341944 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3a078-8d10-4e3a-bc0b-427134e30205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:20.342222 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c3a078-8d10-4e3a-bc0b-427134e30205 admin admin] PATH_INFO: `/v3/projects/c544bafecff7489f925728b91d16a25d/users/b31557d75ca04efb97111df4820601af/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:20.357601 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c3a078-8d10-4e3a-bc0b-427134e30205 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c544bafecff7489f925728b91d16a25d, user_id=b31557d75ca04efb97111df4820601af, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:20.360299 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c3a078-8d10-4e3a-bc0b-427134e30205 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:20.374241 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1375/2769] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:20 2022] PUT /identity/v3/projects/c544bafecff7489f925728b91d16a25d/users/b31557d75ca04efb97111df4820601af/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:20.378231 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5233b-f246-41ef-90f3-4b3af816c1d4 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:20.390531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc5233b-f246-41ef-90f3-4b3af816c1d4 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:20.404502 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3a078-8d10-4e3a-bc0b-427134e30205 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:20.406330 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-94c3a078-8d10-4e3a-bc0b-427134e30205 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:20.446788 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:20.459329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213acfac-ebf7-4652-a677-2f802159376d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:20.460124 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213acfac-ebf7-4652-a677-2f802159376d admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:20.460414 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213acfac-ebf7-4652-a677-2f802159376d admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:20.460727 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213acfac-ebf7-4652-a677-2f802159376d admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:20.461454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213acfac-ebf7-4652-a677-2f802159376d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:20.463096 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213acfac-ebf7-4652-a677-2f802159376d admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:20.473366 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1395/2770] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:08:20.477825 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c3a078-8d10-4e3a-bc0b-427134e30205 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:20.481572 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213acfac-ebf7-4652-a677-2f802159376d admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:20.483491 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4551dd-9118-4664-9ea3-c892cc5a4fde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:20.484381 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4551dd-9118-4664-9ea3-c892cc5a4fde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:20.484663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4551dd-9118-4664-9ea3-c892cc5a4fde admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:20.484941 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4551dd-9118-4664-9ea3-c892cc5a4fde admin admin] PATH_INFO: `/v3/users/f7f68a84de1646e0a647a34e9be17088` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:20.486145 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4551dd-9118-4664-9ea3-c892cc5a4fde admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7f68a84de1646e0a647a34e9be17088)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:20.488117 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4551dd-9118-4664-9ea3-c892cc5a4fde admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:20.492339 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213acfac-ebf7-4652-a677-2f802159376d admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:20.497045 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c1be75-91c1-4968-ae78-510916ed53c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:20.498183 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1be75-91c1-4968-ae78-510916ed53c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:20.498183 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1be75-91c1-4968-ae78-510916ed53c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:20.498479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c1be75-91c1-4968-ae78-510916ed53c1 admin admin] PATH_INFO: `/v3/projects/c544bafecff7489f925728b91d16a25d/users/b31557d75ca04efb97111df4820601af/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:20.500769 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c1be75-91c1-4968-ae78-510916ed53c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c544bafecff7489f925728b91d16a25d, user_id=b31557d75ca04efb97111df4820601af, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:20.503943 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c1be75-91c1-4968-ae78-510916ed53c1 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:20.516328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1396/2771] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:20 2022] PUT /identity/v3/projects/c544bafecff7489f925728b91d16a25d/users/b31557d75ca04efb97111df4820601af/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:20.526329 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490a0f0-a461-490e-b163-4bccb7c49263 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:20.526742 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490a0f0-a461-490e-b163-4bccb7c49263 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:20.527088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3490a0f0-a461-490e-b163-4bccb7c49263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:20.544786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-bc4551dd-9118-4664-9ea3-c892cc5a4fde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7f68a84de1646e0a647a34e9be17088'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:20.545460 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-bc4551dd-9118-4664-9ea3-c892cc5a4fde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7f68a84de1646e0a647a34e9be17088'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:20.553802 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-bc4551dd-9118-4664-9ea3-c892cc5a4fde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7f68a84de1646e0a647a34e9be17088'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:20.554541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-bc4551dd-9118-4664-9ea3-c892cc5a4fde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7f68a84de1646e0a647a34e9be17088'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:20.555704 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-3490a0f0-a461-490e-b163-4bccb7c49263 None None] MFA Rules not processed for user `b31557d75ca04efb97111df4820601af`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:20.556908 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-3490a0f0-a461-490e-b163-4bccb7c49263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:20.568018 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-bc4551dd-9118-4664-9ea3-c892cc5a4fde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7f68a84de1646e0a647a34e9be17088'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:20.583605 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1376/2772] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:20 2022] DELETE /identity/v3/users/f7f68a84de1646e0a647a34e9be17088 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:20.587910 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324547e-f823-415b-bbc0-ffaf2ed9848f None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:20.588205 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324547e-f823-415b-bbc0-ffaf2ed9848f None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:20.588486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1324547e-f823-415b-bbc0-ffaf2ed9848f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:20.604737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-1324547e-f823-415b-bbc0-ffaf2ed9848f None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:20.605940 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-1324547e-f823-415b-bbc0-ffaf2ed9848f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:20.642631 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1397/2773] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:20.653723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490a0f0-a461-490e-b163-4bccb7c49263 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:20.655172 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-3490a0f0-a461-490e-b163-4bccb7c49263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:20.667928 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:20.678833 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1377/2774] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:20.688329 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1324547e-f823-415b-bbc0-ffaf2ed9848f None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:20.689757 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-1324547e-f823-415b-bbc0-ffaf2ed9848f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:20.714792 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3490a0f0-a461-490e-b163-4bccb7c49263 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:20.754352 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1324547e-f823-415b-bbc0-ffaf2ed9848f None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:20.758374 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54afca-5679-4371-b49a-9626516658b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:20.759084 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54afca-5679-4371-b49a-9626516658b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:20.759371 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54afca-5679-4371-b49a-9626516658b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:20.759653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54afca-5679-4371-b49a-9626516658b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:20.765307 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57bd8a-dfaa-4d22-be47-c0c888c46896 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:20.765956 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57bd8a-dfaa-4d22-be47-c0c888c46896 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:20.766243 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57bd8a-dfaa-4d22-be47-c0c888c46896 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:20.766606 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c57bd8a-dfaa-4d22-be47-c0c888c46896 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:20.768378 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-3c57bd8a-dfaa-4d22-be47-c0c888c46896 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:20.769127 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54afca-5679-4371-b49a-9626516658b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:20.770585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54afca-5679-4371-b49a-9626516658b4 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:20.781162 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1378/2775] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:08:20.801719 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:20.837846 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57bd8a-dfaa-4d22-be47-c0c888c46896 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:20.839578 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c57bd8a-dfaa-4d22-be47-c0c888c46896 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:20.850345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1398/2776] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:08:21.116083 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54afca-5679-4371-b49a-9626516658b4 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:21.118533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-ec54afca-5679-4371-b49a-9626516658b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:21.138261 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54afca-5679-4371-b49a-9626516658b4 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:21.151245 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1740b-8302-430e-9626-1030342ed86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:21.152132 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1740b-8302-430e-9626-1030342ed86a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:21.152462 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1740b-8302-430e-9626-1030342ed86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:21.152842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea1740b-8302-430e-9626-1030342ed86a admin admin] PATH_INFO: `/v3/projects/a8311e09374b4c64be1ffa3609b888ea` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:21.157113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea1740b-8302-430e-9626-1030342ed86a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8311e09374b4c64be1ffa3609b888ea)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:21.161348 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea1740b-8302-430e-9626-1030342ed86a admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:21.259220 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0ea1740b-8302-430e-9626-1030342ed86a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8311e09374b4c64be1ffa3609b888ea', 'request_id': 'req-0ea1740b-8302-430e-9626-1030342ed86a'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:21.266189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0ea1740b-8302-430e-9626-1030342ed86a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8311e09374b4c64be1ffa3609b888ea', 'request_id': 'req-0ea1740b-8302-430e-9626-1030342ed86a'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:21.273851 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0ea1740b-8302-430e-9626-1030342ed86a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8311e09374b4c64be1ffa3609b888ea', 'request_id': 'req-0ea1740b-8302-430e-9626-1030342ed86a'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:21.289292 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0ea1740b-8302-430e-9626-1030342ed86a admin admin] The token cache is being invalidate because project a8311e09374b4c64be1ffa3609b888ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:21.289642 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0ea1740b-8302-430e-9626-1030342ed86a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:21.290290 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-0ea1740b-8302-430e-9626-1030342ed86a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:21.291996 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1379/2777] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:21 2022] DELETE /identity/v3/projects/a8311e09374b4c64be1ffa3609b888ea => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:21.296683 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57bd8a-dfaa-4d22-be47-c0c888c46896 service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:21.310039 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c57bd8a-dfaa-4d22-be47-c0c888c46896 service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:21.369547 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a327d3e1-a1d9-4557-a9a0-dfa1886e45e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:21.370506 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327d3e1-a1d9-4557-a9a0-dfa1886e45e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:21.370927 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327d3e1-a1d9-4557-a9a0-dfa1886e45e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:21.371173 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a327d3e1-a1d9-4557-a9a0-dfa1886e45e6 admin admin] PATH_INFO: `/v3/users/4245a74baabc4de4b62b007a3b16903f` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:21.372385 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327d3e1-a1d9-4557-a9a0-dfa1886e45e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4245a74baabc4de4b62b007a3b16903f)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:21.374461 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a327d3e1-a1d9-4557-a9a0-dfa1886e45e6 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:21.424062 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a327d3e1-a1d9-4557-a9a0-dfa1886e45e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4245a74baabc4de4b62b007a3b16903f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:21.436476 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a327d3e1-a1d9-4557-a9a0-dfa1886e45e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4245a74baabc4de4b62b007a3b16903f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:21.438401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a327d3e1-a1d9-4557-a9a0-dfa1886e45e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4245a74baabc4de4b62b007a3b16903f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:21.467468 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a327d3e1-a1d9-4557-a9a0-dfa1886e45e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4245a74baabc4de4b62b007a3b16903f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:21.468378 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-a327d3e1-a1d9-4557-a9a0-dfa1886e45e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4245a74baabc4de4b62b007a3b16903f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:21.487782 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1399/2778] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:21 2022] DELETE /identity/v3/users/4245a74baabc4de4b62b007a3b16903f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:21.768512 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1740b-8302-430e-9626-1030342ed86a admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:21.773954 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-0ea1740b-8302-430e-9626-1030342ed86a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:21.785639 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea1740b-8302-430e-9626-1030342ed86a admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:21.845824 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882741b8-fc7c-42ec-9d9c-9ce90d857c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:21.846720 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882741b8-fc7c-42ec-9d9c-9ce90d857c12 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:21.847012 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882741b8-fc7c-42ec-9d9c-9ce90d857c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:21.847293 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882741b8-fc7c-42ec-9d9c-9ce90d857c12 admin admin] PATH_INFO: `/v3/projects/035da623e1a349e78231b4df697c3c1f` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:21.848511 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882741b8-fc7c-42ec-9d9c-9ce90d857c12 admin admin] RBAC: Authorizing `identity:delete_project(project_id=035da623e1a349e78231b4df697c3c1f)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:21.851759 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882741b8-fc7c-42ec-9d9c-9ce90d857c12 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:21.884055 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:21.901251 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-882741b8-fc7c-42ec-9d9c-9ce90d857c12 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '035da623e1a349e78231b4df697c3c1f', 'request_id': 'req-882741b8-fc7c-42ec-9d9c-9ce90d857c12'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:21.907589 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-882741b8-fc7c-42ec-9d9c-9ce90d857c12 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '035da623e1a349e78231b4df697c3c1f', 'request_id': 'req-882741b8-fc7c-42ec-9d9c-9ce90d857c12'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:21.913996 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-882741b8-fc7c-42ec-9d9c-9ce90d857c12 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '035da623e1a349e78231b4df697c3c1f', 'request_id': 'req-882741b8-fc7c-42ec-9d9c-9ce90d857c12'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:21.927864 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-882741b8-fc7c-42ec-9d9c-9ce90d857c12 admin admin] The token cache is being invalidate because project 035da623e1a349e78231b4df697c3c1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:21.928154 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-882741b8-fc7c-42ec-9d9c-9ce90d857c12 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:21.928790 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-882741b8-fc7c-42ec-9d9c-9ce90d857c12 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:21.930341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1380/2779] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:21 2022] DELETE /identity/v3/projects/035da623e1a349e78231b4df697c3c1f => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:21.935219 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83409c-4f8c-4509-a7bd-e3d5153dc098 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:21.935606 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83409c-4f8c-4509-a7bd-e3d5153dc098 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:21.935897 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a83409c-4f8c-4509-a7bd-e3d5153dc098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:21.957080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-9a83409c-4f8c-4509-a7bd-e3d5153dc098 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:21.958665 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-9a83409c-4f8c-4509-a7bd-e3d5153dc098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:22.006910 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:22.023734 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1400/2780] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:22.032457 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6910f-e4fa-4fdc-ba96-24496881d495 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:22.032739 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6910f-e4fa-4fdc-ba96-24496881d495 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:22.033045 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c6910f-e4fa-4fdc-ba96-24496881d495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:22.052238 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-93c6910f-e4fa-4fdc-ba96-24496881d495 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:22.053811 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-93c6910f-e4fa-4fdc-ba96-24496881d495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:22.066458 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1381/2781] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:22.070673 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83409c-4f8c-4509-a7bd-e3d5153dc098 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:22.080213 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a83409c-4f8c-4509-a7bd-e3d5153dc098 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:22.085447 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29caa9ad-860f-43b2-9e06-a1e5e33e4920 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:22.086217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29caa9ad-860f-43b2-9e06-a1e5e33e4920 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:22.086507 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29caa9ad-860f-43b2-9e06-a1e5e33e4920 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:22.086818 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29caa9ad-860f-43b2-9e06-a1e5e33e4920 admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:22.087551 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29caa9ad-860f-43b2-9e06-a1e5e33e4920 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:08:22.087868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29caa9ad-860f-43b2-9e06-a1e5e33e4920 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:22.089432 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29caa9ad-860f-43b2-9e06-a1e5e33e4920 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:22.102452 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1401/2782] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:08:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:08:22.106846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6910f-e4fa-4fdc-ba96-24496881d495 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:22.116097 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c6910f-e4fa-4fdc-ba96-24496881d495 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:22.120770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ee31e-1912-4528-9805-2e0c4c9db08a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:22.121419 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ee31e-1912-4528-9805-2e0c4c9db08a admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:22.121707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ee31e-1912-4528-9805-2e0c4c9db08a admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:22.121986 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574ee31e-1912-4528-9805-2e0c4c9db08a admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:22.122711 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ee31e-1912-4528-9805-2e0c4c9db08a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:22.124613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574ee31e-1912-4528-9805-2e0c4c9db08a admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:22.150736 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1382/2783] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:22 2022] POST /identity/v3/projects => generated 373 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:22.154862 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29caa9ad-860f-43b2-9e06-a1e5e33e4920 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:22.166060 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29caa9ad-860f-43b2-9e06-a1e5e33e4920 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:22.221783 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31213d75-f235-4466-bace-54ca626785da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:22.222462 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31213d75-f235-4466-bace-54ca626785da admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:22.222793 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31213d75-f235-4466-bace-54ca626785da admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:22.223083 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31213d75-f235-4466-bace-54ca626785da admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:22.223789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31213d75-f235-4466-bace-54ca626785da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:22.226299 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31213d75-f235-4466-bace-54ca626785da admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:22.247324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1402/2784] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:22 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:22.251419 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ee31e-1912-4528-9805-2e0c4c9db08a admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:22.262315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574ee31e-1912-4528-9805-2e0c4c9db08a admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:22.267007 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1c223-8cc2-4881-ae81-8a5ac489761a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:22.267719 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1c223-8cc2-4881-ae81-8a5ac489761a admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:22.268011 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1c223-8cc2-4881-ae81-8a5ac489761a admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:22.268292 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d1c223-8cc2-4881-ae81-8a5ac489761a admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:22.269017 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1c223-8cc2-4881-ae81-8a5ac489761a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:22.270680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d1c223-8cc2-4881-ae81-8a5ac489761a admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:22.289786 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1383/2785] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:22.291697 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31213d75-f235-4466-bace-54ca626785da admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:22.303309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31213d75-f235-4466-bace-54ca626785da admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:22.309584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d0145f-f379-4bc9-95ef-372731c5c16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:22.310445 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0145f-f379-4bc9-95ef-372731c5c16a admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:22.310775 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0145f-f379-4bc9-95ef-372731c5c16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:22.311708 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d0145f-f379-4bc9-95ef-372731c5c16a admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:22.312451 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0145f-f379-4bc9-95ef-372731c5c16a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:22.314472 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d0145f-f379-4bc9-95ef-372731c5c16a admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:22.331475 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1c223-8cc2-4881-ae81-8a5ac489761a admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:22.333596 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1403/2786] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:22 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:08:22.344618 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d1c223-8cc2-4881-ae81-8a5ac489761a admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:22.349368 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabc7a4-58ce-43bb-894d-20416c1ca26f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:22.350177 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabc7a4-58ce-43bb-894d-20416c1ca26f admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:22.350468 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabc7a4-58ce-43bb-894d-20416c1ca26f admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:22.350784 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afabc7a4-58ce-43bb-894d-20416c1ca26f admin admin] PATH_INFO: `/v3/projects/730355ba00924bb59560327be63b9ad7/users/94f130916ee940ecac9f9849eb01025f/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:22.367947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabc7a4-58ce-43bb-894d-20416c1ca26f admin admin] RBAC: Authorizing `identity:create_grant(project_id=730355ba00924bb59560327be63b9ad7, user_id=94f130916ee940ecac9f9849eb01025f, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:22.370619 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afabc7a4-58ce-43bb-894d-20416c1ca26f admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:22.386804 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1384/2787] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:22 2022] PUT /identity/v3/projects/730355ba00924bb59560327be63b9ad7/users/94f130916ee940ecac9f9849eb01025f/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:22.398439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55297a36-032c-4244-be9c-6f26539e01e3 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:22.398439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55297a36-032c-4244-be9c-6f26539e01e3 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:22.398439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55297a36-032c-4244-be9c-6f26539e01e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:22.425450 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-55297a36-032c-4244-be9c-6f26539e01e3 None None] MFA Rules not processed for user `94f130916ee940ecac9f9849eb01025f`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:22.425681 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabc7a4-58ce-43bb-894d-20416c1ca26f admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:22.426736 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-55297a36-032c-4244-be9c-6f26539e01e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:22.427365 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-afabc7a4-58ce-43bb-894d-20416c1ca26f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:22.488211 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afabc7a4-58ce-43bb-894d-20416c1ca26f admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:22.492105 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df66788-e3af-442d-99db-ad0e4d2d0971 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:22.492846 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df66788-e3af-442d-99db-ad0e4d2d0971 service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:22.493214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df66788-e3af-442d-99db-ad0e4d2d0971 service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:22.493421 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df66788-e3af-442d-99db-ad0e4d2d0971 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:22.495061 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1404/2788] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:22.495534 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-4df66788-e3af-442d-99db-ad0e4d2d0971 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:22.506752 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55297a36-032c-4244-be9c-6f26539e01e3 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:22.508403 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-55297a36-032c-4244-be9c-6f26539e01e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:22.521050 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55297a36-032c-4244-be9c-6f26539e01e3 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:22.525398 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72375cc4-3f8f-4fbf-bdbf-3975a13d9c06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:22.526063 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72375cc4-3f8f-4fbf-bdbf-3975a13d9c06 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:22.526351 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72375cc4-3f8f-4fbf-bdbf-3975a13d9c06 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:22.526830 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72375cc4-3f8f-4fbf-bdbf-3975a13d9c06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:22.535610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72375cc4-3f8f-4fbf-bdbf-3975a13d9c06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:22.537249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72375cc4-3f8f-4fbf-bdbf-3975a13d9c06 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:22.546575 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1405/2789] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:22.566144 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72375cc4-3f8f-4fbf-bdbf-3975a13d9c06 service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:22.569653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df66788-e3af-442d-99db-ad0e4d2d0971 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:22.572620 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df66788-e3af-442d-99db-ad0e4d2d0971 service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:22.585012 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1385/2790] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:08:22.611205 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:22.656832 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72375cc4-3f8f-4fbf-bdbf-3975a13d9c06 service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:22.663132 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fef76-ae37-47b2-bc46-dddf4c7dbf42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:22.665320 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fef76-ae37-47b2-bc46-dddf4c7dbf42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:22.665616 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fef76-ae37-47b2-bc46-dddf4c7dbf42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:22.665901 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6fef76-ae37-47b2-bc46-dddf4c7dbf42 admin admin] PATH_INFO: `/v3/users/94f130916ee940ecac9f9849eb01025f` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:22.667771 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6fef76-ae37-47b2-bc46-dddf4c7dbf42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94f130916ee940ecac9f9849eb01025f)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:22.669805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6fef76-ae37-47b2-bc46-dddf4c7dbf42 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:22.741389 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5e6fef76-ae37-47b2-bc46-dddf4c7dbf42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94f130916ee940ecac9f9849eb01025f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:22.757334 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5e6fef76-ae37-47b2-bc46-dddf4c7dbf42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94f130916ee940ecac9f9849eb01025f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:22.758703 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5e6fef76-ae37-47b2-bc46-dddf4c7dbf42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94f130916ee940ecac9f9849eb01025f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:22.781735 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5e6fef76-ae37-47b2-bc46-dddf4c7dbf42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94f130916ee940ecac9f9849eb01025f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:22.782900 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-5e6fef76-ae37-47b2-bc46-dddf4c7dbf42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94f130916ee940ecac9f9849eb01025f'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:22.799777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1406/2791] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:22 2022] DELETE /identity/v3/users/94f130916ee940ecac9f9849eb01025f => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:22.805539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbb555-3b06-4ae0-9a28-d29c1e6876e6 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:22.806020 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbb555-3b06-4ae0-9a28-d29c1e6876e6 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:22.806271 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dbb555-3b06-4ae0-9a28-d29c1e6876e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:22.822200 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:22.830798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-e0dbb555-3b06-4ae0-9a28-d29c1e6876e6 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:22.833337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-e0dbb555-3b06-4ae0-9a28-d29c1e6876e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:22.898314 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1386/2792] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:22.908441 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fef76-ae37-47b2-bc46-dddf4c7dbf42 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:22.909845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5e6fef76-ae37-47b2-bc46-dddf4c7dbf42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:22.966046 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6fef76-ae37-47b2-bc46-dddf4c7dbf42 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:22.970166 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9692d85-1610-481a-8750-be798234ac7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:22.970969 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9692d85-1610-481a-8750-be798234ac7e service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:22.971266 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9692d85-1610-481a-8750-be798234ac7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:22.971571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9692d85-1610-481a-8750-be798234ac7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:22.981234 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9692d85-1610-481a-8750-be798234ac7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:22.982681 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9692d85-1610-481a-8750-be798234ac7e service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:22.992612 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1407/2793] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:23.284840 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbb555-3b06-4ae0-9a28-d29c1e6876e6 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:23.286233 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-e0dbb555-3b06-4ae0-9a28-d29c1e6876e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:23.291350 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9692d85-1610-481a-8750-be798234ac7e service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:23.293147 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b9692d85-1610-481a-8750-be798234ac7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:23.308923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9692d85-1610-481a-8750-be798234ac7e service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:23.313924 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767dcb94-0458-4e73-a3f8-8a5726e15b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:23.314978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dcb94-0458-4e73-a3f8-8a5726e15b6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:23.315309 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dcb94-0458-4e73-a3f8-8a5726e15b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:23.315589 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dcb94-0458-4e73-a3f8-8a5726e15b6e admin admin] PATH_INFO: `/v3/projects/730355ba00924bb59560327be63b9ad7` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:23.316796 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767dcb94-0458-4e73-a3f8-8a5726e15b6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=730355ba00924bb59560327be63b9ad7)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:23.318862 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767dcb94-0458-4e73-a3f8-8a5726e15b6e admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:23.350279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dbb555-3b06-4ae0-9a28-d29c1e6876e6 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:23.354100 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cbfeb3-4979-41eb-8a1c-ab5a632c4216 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:23.354781 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbfeb3-4979-41eb-8a1c-ab5a632c4216 service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:23.355070 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbfeb3-4979-41eb-8a1c-ab5a632c4216 service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:23.355348 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cbfeb3-4979-41eb-8a1c-ab5a632c4216 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:23.357156 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-03cbfeb3-4979-41eb-8a1c-ab5a632c4216 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:23.368637 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-767dcb94-0458-4e73-a3f8-8a5726e15b6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '730355ba00924bb59560327be63b9ad7', 'request_id': 'req-767dcb94-0458-4e73-a3f8-8a5726e15b6e'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:23.369805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:23.376691 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-767dcb94-0458-4e73-a3f8-8a5726e15b6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '730355ba00924bb59560327be63b9ad7', 'request_id': 'req-767dcb94-0458-4e73-a3f8-8a5726e15b6e'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:23.383152 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-767dcb94-0458-4e73-a3f8-8a5726e15b6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '730355ba00924bb59560327be63b9ad7', 'request_id': 'req-767dcb94-0458-4e73-a3f8-8a5726e15b6e'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:23.395570 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-767dcb94-0458-4e73-a3f8-8a5726e15b6e admin admin] The token cache is being invalidate because project 730355ba00924bb59560327be63b9ad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:23.395944 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-767dcb94-0458-4e73-a3f8-8a5726e15b6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:23.397178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-767dcb94-0458-4e73-a3f8-8a5726e15b6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:23.399106 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1408/2794] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:23 2022] DELETE /identity/v3/projects/730355ba00924bb59560327be63b9ad7 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:23.403765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8af16-4d6c-47d5-b891-1dc24af4b150 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:23.404133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8af16-4d6c-47d5-b891-1dc24af4b150 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:23.404488 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b8af16-4d6c-47d5-b891-1dc24af4b150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:23.423212 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a7b8af16-4d6c-47d5-b891-1dc24af4b150 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:23.424615 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a7b8af16-4d6c-47d5-b891-1dc24af4b150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:23.426750 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cbfeb3-4979-41eb-8a1c-ab5a632c4216 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:23.428226 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cbfeb3-4979-41eb-8a1c-ab5a632c4216 service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:23.437955 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1387/2795] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:23 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:23.487634 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1409/2796] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:23.495585 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675fa7e-93db-4b49-aa49-1c7906bdc515 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:23.495989 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675fa7e-93db-4b49-aa49-1c7906bdc515 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:23.496279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0675fa7e-93db-4b49-aa49-1c7906bdc515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:23.514958 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-0675fa7e-93db-4b49-aa49-1c7906bdc515 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:23.516495 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-0675fa7e-93db-4b49-aa49-1c7906bdc515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:23.525726 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1388/2797] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:23.530420 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8af16-4d6c-47d5-b891-1dc24af4b150 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:23.541080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b8af16-4d6c-47d5-b891-1dc24af4b150 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:23.545787 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0115b-a722-4d0a-b0a8-803c4837b442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:23.546481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0115b-a722-4d0a-b0a8-803c4837b442 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:23.546802 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0115b-a722-4d0a-b0a8-803c4837b442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:23.547086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0115b-a722-4d0a-b0a8-803c4837b442 admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:23.547819 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0115b-a722-4d0a-b0a8-803c4837b442 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:08:23.548133 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0115b-a722-4d0a-b0a8-803c4837b442 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:23.549702 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0115b-a722-4d0a-b0a8-803c4837b442 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:23.561373 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1410/2798] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:08:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:08:23.565568 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0675fa7e-93db-4b49-aa49-1c7906bdc515 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:23.575153 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0675fa7e-93db-4b49-aa49-1c7906bdc515 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:23.579900 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deaf07d-912e-42ab-88fa-519d531c41c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:23.580545 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaf07d-912e-42ab-88fa-519d531c41c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:23.580829 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaf07d-912e-42ab-88fa-519d531c41c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:23.581106 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deaf07d-912e-42ab-88fa-519d531c41c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:23.581779 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deaf07d-912e-42ab-88fa-519d531c41c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:23.583702 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deaf07d-912e-42ab-88fa-519d531c41c8 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:23.610533 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1389/2799] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:23 2022] POST /identity/v3/projects => generated 375 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:23.614995 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0115b-a722-4d0a-b0a8-803c4837b442 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:23.625215 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0115b-a722-4d0a-b0a8-803c4837b442 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:23.683555 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71fd41-5017-4e6c-8a10-a67920445c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:23.684292 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71fd41-5017-4e6c-8a10-a67920445c07 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:23.684585 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71fd41-5017-4e6c-8a10-a67920445c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:23.684865 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c71fd41-5017-4e6c-8a10-a67920445c07 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:23.685549 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71fd41-5017-4e6c-8a10-a67920445c07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:23.687714 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c71fd41-5017-4e6c-8a10-a67920445c07 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:23.710145 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1411/2800] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:23 2022] POST /identity/v3/users => generated 288 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 23:08:23.712463 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deaf07d-912e-42ab-88fa-519d531c41c8 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:23.723466 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deaf07d-912e-42ab-88fa-519d531c41c8 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:23.728571 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d130b5f-0b3d-4be8-b028-4bdf46777ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:23.729230 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d130b5f-0b3d-4be8-b028-4bdf46777ebb admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:23.729518 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d130b5f-0b3d-4be8-b028-4bdf46777ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:23.729798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d130b5f-0b3d-4be8-b028-4bdf46777ebb admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:23.730510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d130b5f-0b3d-4be8-b028-4bdf46777ebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:23.732184 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d130b5f-0b3d-4be8-b028-4bdf46777ebb admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:23.742112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1390/2801] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:23.746097 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71fd41-5017-4e6c-8a10-a67920445c07 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:23.756457 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c71fd41-5017-4e6c-8a10-a67920445c07 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:23.761670 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae895de-8441-4e50-a644-eb37f30be710 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:23.762432 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae895de-8441-4e50-a644-eb37f30be710 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:23.762762 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae895de-8441-4e50-a644-eb37f30be710 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:23.763059 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae895de-8441-4e50-a644-eb37f30be710 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:23.763790 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae895de-8441-4e50-a644-eb37f30be710 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:23.765390 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae895de-8441-4e50-a644-eb37f30be710 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:23.779806 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1412/2802] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:23.780984 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d130b5f-0b3d-4be8-b028-4bdf46777ebb admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:23.791291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d130b5f-0b3d-4be8-b028-4bdf46777ebb admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:23.795944 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2de7b-2178-4ce3-8037-5a91c19e6b6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:23.796841 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2de7b-2178-4ce3-8037-5a91c19e6b6a admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:23.797117 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2de7b-2178-4ce3-8037-5a91c19e6b6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:23.797401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2de7b-2178-4ce3-8037-5a91c19e6b6a admin admin] PATH_INFO: `/v3/projects/2384293fd40643bf8e10e0ffefebc9b0/users/421786c2e7054bb8b6ee68322f6d0693/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:23.811943 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2de7b-2178-4ce3-8037-5a91c19e6b6a admin admin] RBAC: Authorizing `identity:create_grant(project_id=2384293fd40643bf8e10e0ffefebc9b0, user_id=421786c2e7054bb8b6ee68322f6d0693, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:23.814554 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2de7b-2178-4ce3-8037-5a91c19e6b6a admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:23.816845 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:23.829821 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1391/2803] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:23 2022] PUT /identity/v3/projects/2384293fd40643bf8e10e0ffefebc9b0/users/421786c2e7054bb8b6ee68322f6d0693/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:23.842129 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cb307-0360-4f2f-b139-73d7d6fd3cea None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:23.842978 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cb307-0360-4f2f-b139-73d7d6fd3cea None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:23.843101 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8cb307-0360-4f2f-b139-73d7d6fd3cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:23.872308 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-fb8cb307-0360-4f2f-b139-73d7d6fd3cea None None] MFA Rules not processed for user `421786c2e7054bb8b6ee68322f6d0693`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:23.874004 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-fb8cb307-0360-4f2f-b139-73d7d6fd3cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:23.909679 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:23.948725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1413/2804] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:23.960297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2de7b-2178-4ce3-8037-5a91c19e6b6a admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:23.962090 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-e4e2de7b-2178-4ce3-8037-5a91c19e6b6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:24.020724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2de7b-2178-4ce3-8037-5a91c19e6b6a admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:24.024890 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce36804-3fdc-4c18-b082-36381fbf952a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:24.025535 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce36804-3fdc-4c18-b082-36381fbf952a service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:24.025820 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce36804-3fdc-4c18-b082-36381fbf952a service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:24.026099 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce36804-3fdc-4c18-b082-36381fbf952a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:24.034421 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce36804-3fdc-4c18-b082-36381fbf952a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:24.035971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce36804-3fdc-4c18-b082-36381fbf952a service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:24.045730 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1392/2805] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:26.048842 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cb307-0360-4f2f-b139-73d7d6fd3cea None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:26.050228 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-fb8cb307-0360-4f2f-b139-73d7d6fd3cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:26.127935 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8cb307-0360-4f2f-b139-73d7d6fd3cea None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:26.133125 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25f560-3db5-4996-8c61-a315c603fcc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:26.134007 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25f560-3db5-4996-8c61-a315c603fcc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:26.134278 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25f560-3db5-4996-8c61-a315c603fcc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:26.134556 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be25f560-3db5-4996-8c61-a315c603fcc4 admin admin] PATH_INFO: `/v3/users/bb33f5a8bce2481c896c9307338ca634` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:26.135772 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25f560-3db5-4996-8c61-a315c603fcc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb33f5a8bce2481c896c9307338ca634)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:26.137671 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be25f560-3db5-4996-8c61-a315c603fcc4 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:26.190378 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-be25f560-3db5-4996-8c61-a315c603fcc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb33f5a8bce2481c896c9307338ca634'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:26.198373 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-be25f560-3db5-4996-8c61-a315c603fcc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb33f5a8bce2481c896c9307338ca634'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:26.199095 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-be25f560-3db5-4996-8c61-a315c603fcc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb33f5a8bce2481c896c9307338ca634'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:26.211710 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-be25f560-3db5-4996-8c61-a315c603fcc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb33f5a8bce2481c896c9307338ca634'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:26.212418 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-be25f560-3db5-4996-8c61-a315c603fcc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb33f5a8bce2481c896c9307338ca634'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:26.227580 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1414/2806] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:26 2022] DELETE /identity/v3/users/bb33f5a8bce2481c896c9307338ca634 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:26.232597 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e215c-9a09-4696-bee9-fee112aa445b None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:26.233420 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e215c-9a09-4696-bee9-fee112aa445b None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:26.233709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e215c-9a09-4696-bee9-fee112aa445b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:26.251593 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-487e215c-9a09-4696-bee9-fee112aa445b None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:26.252822 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-487e215c-9a09-4696-bee9-fee112aa445b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:26.313491 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1393/2807] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:26.323531 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25f560-3db5-4996-8c61-a315c603fcc4 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:26.324940 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-be25f560-3db5-4996-8c61-a315c603fcc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:26.382693 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be25f560-3db5-4996-8c61-a315c603fcc4 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:26.389905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172f135-5629-4fb9-992b-cd43bfb14afe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:26.389905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172f135-5629-4fb9-992b-cd43bfb14afe service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:26.389905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172f135-5629-4fb9-992b-cd43bfb14afe service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:26.389905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172f135-5629-4fb9-992b-cd43bfb14afe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:26.396754 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:26.399674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172f135-5629-4fb9-992b-cd43bfb14afe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:26.402865 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172f135-5629-4fb9-992b-cd43bfb14afe service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:26.415067 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1415/2808] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:26.692084 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e215c-9a09-4696-bee9-fee112aa445b None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:26.693503 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-487e215c-9a09-4696-bee9-fee112aa445b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:26.705653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e215c-9a09-4696-bee9-fee112aa445b None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:26.711814 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273503e1-fcba-40ce-bf9c-ba2563d7171e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:26.712691 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273503e1-fcba-40ce-bf9c-ba2563d7171e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:26.712975 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273503e1-fcba-40ce-bf9c-ba2563d7171e admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:26.713256 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273503e1-fcba-40ce-bf9c-ba2563d7171e admin admin] PATH_INFO: `/v3/projects/5e1ec76367d34267a52ac7921ee12730` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:26.715276 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273503e1-fcba-40ce-bf9c-ba2563d7171e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e1ec76367d34267a52ac7921ee12730)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:26.717196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273503e1-fcba-40ce-bf9c-ba2563d7171e admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:26.765101 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-273503e1-fcba-40ce-bf9c-ba2563d7171e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e1ec76367d34267a52ac7921ee12730', 'request_id': 'req-273503e1-fcba-40ce-bf9c-ba2563d7171e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:26.771741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-273503e1-fcba-40ce-bf9c-ba2563d7171e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e1ec76367d34267a52ac7921ee12730', 'request_id': 'req-273503e1-fcba-40ce-bf9c-ba2563d7171e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:26.777364 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-273503e1-fcba-40ce-bf9c-ba2563d7171e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e1ec76367d34267a52ac7921ee12730', 'request_id': 'req-273503e1-fcba-40ce-bf9c-ba2563d7171e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:26.789208 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-273503e1-fcba-40ce-bf9c-ba2563d7171e admin admin] The token cache is being invalidate because project 5e1ec76367d34267a52ac7921ee12730 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:26.789497 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-273503e1-fcba-40ce-bf9c-ba2563d7171e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:26.790120 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-273503e1-fcba-40ce-bf9c-ba2563d7171e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:26.791690 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1394/2809] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:26 2022] DELETE /identity/v3/projects/5e1ec76367d34267a52ac7921ee12730 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:29.422526 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172f135-5629-4fb9-992b-cd43bfb14afe service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:29.424058 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-5172f135-5629-4fb9-992b-cd43bfb14afe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:29.441470 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172f135-5629-4fb9-992b-cd43bfb14afe service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:29.510459 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cf7c5-2bec-42fd-929c-0da2b6e62243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:29.511578 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cf7c5-2bec-42fd-929c-0da2b6e62243 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:29.511937 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cf7c5-2bec-42fd-929c-0da2b6e62243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:29.512244 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6cf7c5-2bec-42fd-929c-0da2b6e62243 admin admin] PATH_INFO: `/v3/users/b23c891e97bd459a96502be15ae24396` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:29.513723 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6cf7c5-2bec-42fd-929c-0da2b6e62243 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b23c891e97bd459a96502be15ae24396)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:29.515700 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6cf7c5-2bec-42fd-929c-0da2b6e62243 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:29.563642 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-dc6cf7c5-2bec-42fd-929c-0da2b6e62243 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b23c891e97bd459a96502be15ae24396'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:29.572408 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-dc6cf7c5-2bec-42fd-929c-0da2b6e62243 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b23c891e97bd459a96502be15ae24396'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:29.573322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-dc6cf7c5-2bec-42fd-929c-0da2b6e62243 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b23c891e97bd459a96502be15ae24396'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:29.587276 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-dc6cf7c5-2bec-42fd-929c-0da2b6e62243 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b23c891e97bd459a96502be15ae24396'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:29.588121 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-dc6cf7c5-2bec-42fd-929c-0da2b6e62243 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b23c891e97bd459a96502be15ae24396'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:29.602053 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1416/2810] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:29 2022] DELETE /identity/v3/users/b23c891e97bd459a96502be15ae24396 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:08:29.606699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049eb15e-2035-4e0a-90d9-7eeeab055221 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:29.607360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049eb15e-2035-4e0a-90d9-7eeeab055221 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:29.607360 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049eb15e-2035-4e0a-90d9-7eeeab055221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:29.626162 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-049eb15e-2035-4e0a-90d9-7eeeab055221 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:29.627436 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-049eb15e-2035-4e0a-90d9-7eeeab055221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:29.642378 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:29.721130 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1395/2811] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:29.734657 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cf7c5-2bec-42fd-929c-0da2b6e62243 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:29.736487 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-dc6cf7c5-2bec-42fd-929c-0da2b6e62243 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:29.826222 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6cf7c5-2bec-42fd-929c-0da2b6e62243 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:29.830209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d315b-dab3-4342-89ea-f965e6c2d266 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:29.830915 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d315b-dab3-4342-89ea-f965e6c2d266 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:29.831206 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d315b-dab3-4342-89ea-f965e6c2d266 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:29.831481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913d315b-dab3-4342-89ea-f965e6c2d266 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:29.844123 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d315b-dab3-4342-89ea-f965e6c2d266 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:29.845765 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913d315b-dab3-4342-89ea-f965e6c2d266 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:29.862873 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1417/2812] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:30.168593 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049eb15e-2035-4e0a-90d9-7eeeab055221 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:30.180074 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-049eb15e-2035-4e0a-90d9-7eeeab055221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:30.193515 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049eb15e-2035-4e0a-90d9-7eeeab055221 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:30.199018 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd5525-ff85-428f-b758-d075df052adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:30.199991 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd5525-ff85-428f-b758-d075df052adf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:30.200375 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd5525-ff85-428f-b758-d075df052adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:30.200670 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd5525-ff85-428f-b758-d075df052adf admin admin] PATH_INFO: `/v3/projects/2a78e95dc9fe48e3a421d3e0d7138a9e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:30.201886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd5525-ff85-428f-b758-d075df052adf admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a78e95dc9fe48e3a421d3e0d7138a9e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:30.203830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81dd5525-ff85-428f-b758-d075df052adf admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:30.254414 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-81dd5525-ff85-428f-b758-d075df052adf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a78e95dc9fe48e3a421d3e0d7138a9e', 'request_id': 'req-81dd5525-ff85-428f-b758-d075df052adf'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:30.261520 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-81dd5525-ff85-428f-b758-d075df052adf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a78e95dc9fe48e3a421d3e0d7138a9e', 'request_id': 'req-81dd5525-ff85-428f-b758-d075df052adf'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:30.267005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-81dd5525-ff85-428f-b758-d075df052adf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a78e95dc9fe48e3a421d3e0d7138a9e', 'request_id': 'req-81dd5525-ff85-428f-b758-d075df052adf'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:30.279663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-81dd5525-ff85-428f-b758-d075df052adf admin admin] The token cache is being invalidate because project 2a78e95dc9fe48e3a421d3e0d7138a9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:30.279956 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-81dd5525-ff85-428f-b758-d075df052adf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:30.280575 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-81dd5525-ff85-428f-b758-d075df052adf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:30.282131 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1396/2813] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:30 2022] DELETE /identity/v3/projects/2a78e95dc9fe48e3a421d3e0d7138a9e => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:30.286092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d315b-dab3-4342-89ea-f965e6c2d266 service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:30.295920 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913d315b-dab3-4342-89ea-f965e6c2d266 service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:30.308365 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:30.353541 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca82ab-13aa-4f63-8401-6aa30b46e939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:30.354423 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca82ab-13aa-4f63-8401-6aa30b46e939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:30.354740 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca82ab-13aa-4f63-8401-6aa30b46e939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:30.355019 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbca82ab-13aa-4f63-8401-6aa30b46e939 admin admin] PATH_INFO: `/v3/users/1b14014a3885495089c46346211ffe20` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:30.356223 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca82ab-13aa-4f63-8401-6aa30b46e939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b14014a3885495089c46346211ffe20)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:30.358116 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbca82ab-13aa-4f63-8401-6aa30b46e939 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:30.403191 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-fbca82ab-13aa-4f63-8401-6aa30b46e939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b14014a3885495089c46346211ffe20'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:30.409845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-fbca82ab-13aa-4f63-8401-6aa30b46e939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b14014a3885495089c46346211ffe20'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:30.410495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-fbca82ab-13aa-4f63-8401-6aa30b46e939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b14014a3885495089c46346211ffe20'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:30.421981 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-fbca82ab-13aa-4f63-8401-6aa30b46e939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b14014a3885495089c46346211ffe20'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:30.422627 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-fbca82ab-13aa-4f63-8401-6aa30b46e939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b14014a3885495089c46346211ffe20'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:30.435808 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1418/2814] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:30 2022] DELETE /identity/v3/users/1b14014a3885495089c46346211ffe20 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:30.711363 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd5525-ff85-428f-b758-d075df052adf admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:30.712822 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-81dd5525-ff85-428f-b758-d075df052adf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:30.726987 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81dd5525-ff85-428f-b758-d075df052adf admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:30.801516 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbccacd-d033-4359-9285-205c0ebc363f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:30.802996 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbccacd-d033-4359-9285-205c0ebc363f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:30.803279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbccacd-d033-4359-9285-205c0ebc363f admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:30.803724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbccacd-d033-4359-9285-205c0ebc363f admin admin] PATH_INFO: `/v3/projects/693f0e1850234fe69e386e6b18155a1e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:30.805012 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbccacd-d033-4359-9285-205c0ebc363f admin admin] RBAC: Authorizing `identity:delete_project(project_id=693f0e1850234fe69e386e6b18155a1e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:30.807048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbccacd-d033-4359-9285-205c0ebc363f admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:30.855530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:30.858901 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-dcbccacd-d033-4359-9285-205c0ebc363f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '693f0e1850234fe69e386e6b18155a1e', 'request_id': 'req-dcbccacd-d033-4359-9285-205c0ebc363f'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:30.866873 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-dcbccacd-d033-4359-9285-205c0ebc363f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '693f0e1850234fe69e386e6b18155a1e', 'request_id': 'req-dcbccacd-d033-4359-9285-205c0ebc363f'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:30.873517 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-dcbccacd-d033-4359-9285-205c0ebc363f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '693f0e1850234fe69e386e6b18155a1e', 'request_id': 'req-dcbccacd-d033-4359-9285-205c0ebc363f'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:30.886317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-dcbccacd-d033-4359-9285-205c0ebc363f admin admin] The token cache is being invalidate because project 693f0e1850234fe69e386e6b18155a1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:30.886772 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-dcbccacd-d033-4359-9285-205c0ebc363f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:30.887548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-dcbccacd-d033-4359-9285-205c0ebc363f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:30.889810 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1397/2815] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:30 2022] DELETE /identity/v3/projects/693f0e1850234fe69e386e6b18155a1e => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:30.894086 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca82ab-13aa-4f63-8401-6aa30b46e939 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:30.896037 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-fbca82ab-13aa-4f63-8401-6aa30b46e939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:30.909137 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbca82ab-13aa-4f63-8401-6aa30b46e939 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:30.965376 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd3903-a684-470b-aa93-d77c9afd3182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:30.966252 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd3903-a684-470b-aa93-d77c9afd3182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:30.966537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd3903-a684-470b-aa93-d77c9afd3182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:30.966845 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd3903-a684-470b-aa93-d77c9afd3182 admin admin] PATH_INFO: `/v3/users/b31557d75ca04efb97111df4820601af` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:30.968016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd3903-a684-470b-aa93-d77c9afd3182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b31557d75ca04efb97111df4820601af)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:30.969912 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cd3903-a684-470b-aa93-d77c9afd3182 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:31.009334 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:31.016327 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-26cd3903-a684-470b-aa93-d77c9afd3182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b31557d75ca04efb97111df4820601af'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:31.023649 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-26cd3903-a684-470b-aa93-d77c9afd3182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b31557d75ca04efb97111df4820601af'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:31.024408 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-26cd3903-a684-470b-aa93-d77c9afd3182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b31557d75ca04efb97111df4820601af'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:31.036089 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-26cd3903-a684-470b-aa93-d77c9afd3182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b31557d75ca04efb97111df4820601af'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:31.036755 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-26cd3903-a684-470b-aa93-d77c9afd3182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b31557d75ca04efb97111df4820601af'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:31.055677 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1419/2816] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:30 2022] DELETE /identity/v3/users/b31557d75ca04efb97111df4820601af => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:31.336555 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbccacd-d033-4359-9285-205c0ebc363f admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:31.337972 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-dcbccacd-d033-4359-9285-205c0ebc363f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:31.351871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbccacd-d033-4359-9285-205c0ebc363f admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:31.411813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5068ce6b-23d7-4a83-acdd-fddc689597c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:31.412659 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068ce6b-23d7-4a83-acdd-fddc689597c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:31.412970 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068ce6b-23d7-4a83-acdd-fddc689597c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:31.413246 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5068ce6b-23d7-4a83-acdd-fddc689597c4 admin admin] PATH_INFO: `/v3/projects/c544bafecff7489f925728b91d16a25d` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:31.414659 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5068ce6b-23d7-4a83-acdd-fddc689597c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c544bafecff7489f925728b91d16a25d)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:31.416601 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5068ce6b-23d7-4a83-acdd-fddc689597c4 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:31.471142 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5068ce6b-23d7-4a83-acdd-fddc689597c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c544bafecff7489f925728b91d16a25d', 'request_id': 'req-5068ce6b-23d7-4a83-acdd-fddc689597c4'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:31.479462 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5068ce6b-23d7-4a83-acdd-fddc689597c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c544bafecff7489f925728b91d16a25d', 'request_id': 'req-5068ce6b-23d7-4a83-acdd-fddc689597c4'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:31.485965 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5068ce6b-23d7-4a83-acdd-fddc689597c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c544bafecff7489f925728b91d16a25d', 'request_id': 'req-5068ce6b-23d7-4a83-acdd-fddc689597c4'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:31.499149 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5068ce6b-23d7-4a83-acdd-fddc689597c4 admin admin] The token cache is being invalidate because project c544bafecff7489f925728b91d16a25d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:31.499438 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5068ce6b-23d7-4a83-acdd-fddc689597c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:31.500108 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-5068ce6b-23d7-4a83-acdd-fddc689597c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:31.501708 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1398/2817] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:31 2022] DELETE /identity/v3/projects/c544bafecff7489f925728b91d16a25d => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:31.507238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e6228-0c57-4be2-91e9-5176223c570d None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:31.507724 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e6228-0c57-4be2-91e9-5176223c570d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:31.507836 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820e6228-0c57-4be2-91e9-5176223c570d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:31.526804 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-820e6228-0c57-4be2-91e9-5176223c570d None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:31.528006 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-820e6228-0c57-4be2-91e9-5176223c570d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:31.587646 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1420/2818] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:31.595317 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47faf0-52bf-4137-ade6-32cf64e26cea None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:31.595614 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47faf0-52bf-4137-ade6-32cf64e26cea None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:31.595898 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed47faf0-52bf-4137-ade6-32cf64e26cea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:31.614790 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-ed47faf0-52bf-4137-ade6-32cf64e26cea None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:31.616011 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-ed47faf0-52bf-4137-ade6-32cf64e26cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:31.623968 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1399/2819] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:31.628180 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e6228-0c57-4be2-91e9-5176223c570d None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:31.638233 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820e6228-0c57-4be2-91e9-5176223c570d None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:31.642982 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dec095-6fc0-4999-8299-676929dc628c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:31.643639 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dec095-6fc0-4999-8299-676929dc628c admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:31.643934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dec095-6fc0-4999-8299-676929dc628c admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:31.644212 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dec095-6fc0-4999-8299-676929dc628c admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:31.644939 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dec095-6fc0-4999-8299-676929dc628c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:08:31.645251 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dec095-6fc0-4999-8299-676929dc628c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:31.647381 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dec095-6fc0-4999-8299-676929dc628c admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:31.659895 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1421/2820] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:08:31.664684 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed47faf0-52bf-4137-ade6-32cf64e26cea None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:31.674774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed47faf0-52bf-4137-ade6-32cf64e26cea None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:31.679678 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fde0d-9dd2-4c19-b562-eb87f1f29f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:31.680391 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fde0d-9dd2-4c19-b562-eb87f1f29f47 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:31.680676 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fde0d-9dd2-4c19-b562-eb87f1f29f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:31.680979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fde0d-9dd2-4c19-b562-eb87f1f29f47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:31.681693 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fde0d-9dd2-4c19-b562-eb87f1f29f47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:31.683755 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fde0d-9dd2-4c19-b562-eb87f1f29f47 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:31.710076 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1400/2821] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:31 2022] POST /identity/v3/projects => generated 367 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:31.714785 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dec095-6fc0-4999-8299-676929dc628c admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:31.729587 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dec095-6fc0-4999-8299-676929dc628c admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:31.790948 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6876b7-5034-4e98-8e85-6b4369ddc34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:31.791626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6876b7-5034-4e98-8e85-6b4369ddc34c admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:31.791917 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6876b7-5034-4e98-8e85-6b4369ddc34c admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:31.792200 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6876b7-5034-4e98-8e85-6b4369ddc34c admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:31.792888 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6876b7-5034-4e98-8e85-6b4369ddc34c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:31.794829 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6876b7-5034-4e98-8e85-6b4369ddc34c admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:31.814274 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1422/2822] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:31 2022] POST /identity/v3/users => generated 284 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:31.818741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fde0d-9dd2-4c19-b562-eb87f1f29f47 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:31.829346 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fde0d-9dd2-4c19-b562-eb87f1f29f47 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:31.834625 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ac069-c925-44f5-8791-130857f4a6d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:31.835307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ac069-c925-44f5-8791-130857f4a6d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:31.835633 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ac069-c925-44f5-8791-130857f4a6d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:31.835917 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458ac069-c925-44f5-8791-130857f4a6d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:31.836642 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ac069-c925-44f5-8791-130857f4a6d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:31.838624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458ac069-c925-44f5-8791-130857f4a6d7 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:31.850315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1401/2823] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:08:31.854427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6876b7-5034-4e98-8e85-6b4369ddc34c admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:31.866028 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6876b7-5034-4e98-8e85-6b4369ddc34c admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:31.872172 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e02eb8f-8c4f-4e40-b4a6-3531a43b4af8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:31.873030 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02eb8f-8c4f-4e40-b4a6-3531a43b4af8 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:31.873246 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02eb8f-8c4f-4e40-b4a6-3531a43b4af8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:31.873594 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e02eb8f-8c4f-4e40-b4a6-3531a43b4af8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:31.874456 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e02eb8f-8c4f-4e40-b4a6-3531a43b4af8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:31.876534 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e02eb8f-8c4f-4e40-b4a6-3531a43b4af8 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:31.889794 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1423/2824] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:31.894521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ac069-c925-44f5-8791-130857f4a6d7 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:31.906878 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458ac069-c925-44f5-8791-130857f4a6d7 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:31.913137 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44449a8d-f2a7-4f4e-9011-5c478491fee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:31.914082 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44449a8d-f2a7-4f4e-9011-5c478491fee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:31.914368 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44449a8d-f2a7-4f4e-9011-5c478491fee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:31.914662 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44449a8d-f2a7-4f4e-9011-5c478491fee0 admin admin] PATH_INFO: `/v3/projects/5fe64a5c67c541419efe1d817ec5c26b/users/c50aaf9a551d4f2e8ada9ba762f337aa/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:31.929780 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44449a8d-f2a7-4f4e-9011-5c478491fee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fe64a5c67c541419efe1d817ec5c26b, user_id=c50aaf9a551d4f2e8ada9ba762f337aa, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:31.932473 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44449a8d-f2a7-4f4e-9011-5c478491fee0 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:31.941967 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:31.947739 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1402/2825] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:31 2022] PUT /identity/v3/projects/5fe64a5c67c541419efe1d817ec5c26b/users/c50aaf9a551d4f2e8ada9ba762f337aa/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:32.020598 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1280d74-9248-49af-ac7c-f966d10a8b0d None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:32.021521 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1280d74-9248-49af-ac7c-f966d10a8b0d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:32.021806 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1280d74-9248-49af-ac7c-f966d10a8b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:32.049073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-a1280d74-9248-49af-ac7c-f966d10a8b0d None None] MFA Rules not processed for user `c50aaf9a551d4f2e8ada9ba762f337aa`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:32.050294 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a1280d74-9248-49af-ac7c-f966d10a8b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:32.059462 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:32.117023 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1424/2826] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:32.128868 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44449a8d-f2a7-4f4e-9011-5c478491fee0 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:32.131329 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-44449a8d-f2a7-4f4e-9011-5c478491fee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:32.191252 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44449a8d-f2a7-4f4e-9011-5c478491fee0 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:32.195718 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971fea89-2c8e-4884-9f51-d30fea71ecf5 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915f4296504b47d796e3a5b99550eb75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:32.196369 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fea89-2c8e-4884-9f51-d30fea71ecf5 service glance] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:32.196660 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fea89-2c8e-4884-9f51-d30fea71ecf5 service glance] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:32.196945 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971fea89-2c8e-4884-9f51-d30fea71ecf5 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:32.205665 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fea89-2c8e-4884-9f51-d30fea71ecf5 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:32.207300 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-971fea89-2c8e-4884-9f51-d30fea71ecf5 service glance] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:32.217510 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1403/2827] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:32 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:42.114001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=22 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:42.213248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=26 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:48.727232 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1280d74-9248-49af-ac7c-f966d10a8b0d None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:48.728682 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a1280d74-9248-49af-ac7c-f966d10a8b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:48.799420 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1280d74-9248-49af-ac7c-f966d10a8b0d None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:48.804442 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e8401-9ca9-4361-9c34-96e7462130e1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9593939641c4c7797c1981206b40e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:48.805097 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e8401-9ca9-4361-9c34-96e7462130e1 service placement] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:48.805390 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e8401-9ca9-4361-9c34-96e7462130e1 service placement] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:48.805674 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843e8401-9ca9-4361-9c34-96e7462130e1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:48.807508 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-843e8401-9ca9-4361-9c34-96e7462130e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:48.820797 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e8401-9ca9-4361-9c34-96e7462130e1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:48.822409 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843e8401-9ca9-4361-9c34-96e7462130e1 service placement] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:48.834028 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1425/2828] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:48 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:50.882048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeff38c-f858-43f2-b69b-78079be59d0e None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:50.882511 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeff38c-f858-43f2-b69b-78079be59d0e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:50.882650 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eeff38c-f858-43f2-b69b-78079be59d0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:50.900646 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-0eeff38c-f858-43f2-b69b-78079be59d0e None None] MFA Rules not processed for user `2697aa77265e4f5783f87ffb3d4992b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:50.901842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-0eeff38c-f858-43f2-b69b-78079be59d0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:50.975316 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1404/2829] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:51.083316 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e8401-9ca9-4361-9c34-96e7462130e1 service placement] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:51.084968 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-843e8401-9ca9-4361-9c34-96e7462130e1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:51.160224 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843e8401-9ca9-4361-9c34-96e7462130e1 service placement] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:51.165590 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b403ce-1535-430e-b0c5-14e3acc2a288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:51.166561 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b403ce-1535-430e-b0c5-14e3acc2a288 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:51.167092 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b403ce-1535-430e-b0c5-14e3acc2a288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:51.167615 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b403ce-1535-430e-b0c5-14e3acc2a288 admin admin] PATH_INFO: `/v3/users/421786c2e7054bb8b6ee68322f6d0693` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:51.169016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b403ce-1535-430e-b0c5-14e3acc2a288 admin admin] RBAC: Authorizing `identity:delete_user(user_id=421786c2e7054bb8b6ee68322f6d0693)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:51.171126 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b403ce-1535-430e-b0c5-14e3acc2a288 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:51.222763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-73b403ce-1535-430e-b0c5-14e3acc2a288 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '421786c2e7054bb8b6ee68322f6d0693'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:51.236372 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-73b403ce-1535-430e-b0c5-14e3acc2a288 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '421786c2e7054bb8b6ee68322f6d0693'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:51.237643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-73b403ce-1535-430e-b0c5-14e3acc2a288 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '421786c2e7054bb8b6ee68322f6d0693'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:51.251830 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-73b403ce-1535-430e-b0c5-14e3acc2a288 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '421786c2e7054bb8b6ee68322f6d0693'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:51.252146 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-73b403ce-1535-430e-b0c5-14e3acc2a288 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '421786c2e7054bb8b6ee68322f6d0693'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:51.267104 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1426/2830] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:51 2022] DELETE /identity/v3/users/421786c2e7054bb8b6ee68322f6d0693 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:51.271919 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6528d9-121f-4cf7-9f36-8d9150f5ba95 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:51.272216 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6528d9-121f-4cf7-9f36-8d9150f5ba95 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:51.272498 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6528d9-121f-4cf7-9f36-8d9150f5ba95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:51.290312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-fa6528d9-121f-4cf7-9f36-8d9150f5ba95 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:51.291596 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-fa6528d9-121f-4cf7-9f36-8d9150f5ba95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:51.354539 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1405/2831] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:51.366015 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b403ce-1535-430e-b0c5-14e3acc2a288 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:51.367643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-73b403ce-1535-430e-b0c5-14e3acc2a288 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:51.379805 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:51.432286 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b403ce-1535-430e-b0c5-14e3acc2a288 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:51.435993 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea64b39e-50c3-4ad1-ace8-c76cfb193be4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:51.436675 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64b39e-50c3-4ad1-ace8-c76cfb193be4 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:51.436964 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64b39e-50c3-4ad1-ace8-c76cfb193be4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:51.437244 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea64b39e-50c3-4ad1-ace8-c76cfb193be4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:51.448870 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64b39e-50c3-4ad1-ace8-c76cfb193be4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:51.450334 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea64b39e-50c3-4ad1-ace8-c76cfb193be4 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:51.462980 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1427/2832] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:51.757029 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6528d9-121f-4cf7-9f36-8d9150f5ba95 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:51.759058 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-fa6528d9-121f-4cf7-9f36-8d9150f5ba95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:51.787842 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6528d9-121f-4cf7-9f36-8d9150f5ba95 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:51.795382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375d8c50-57a3-4da0-8ac2-c16adc57cee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:51.796260 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d8c50-57a3-4da0-8ac2-c16adc57cee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:51.796663 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d8c50-57a3-4da0-8ac2-c16adc57cee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:51.797068 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375d8c50-57a3-4da0-8ac2-c16adc57cee0 admin admin] PATH_INFO: `/v3/projects/2384293fd40643bf8e10e0ffefebc9b0` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:51.798331 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d8c50-57a3-4da0-8ac2-c16adc57cee0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2384293fd40643bf8e10e0ffefebc9b0)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:51.800441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375d8c50-57a3-4da0-8ac2-c16adc57cee0 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:51.883069 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-375d8c50-57a3-4da0-8ac2-c16adc57cee0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2384293fd40643bf8e10e0ffefebc9b0', 'request_id': 'req-375d8c50-57a3-4da0-8ac2-c16adc57cee0'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:51.902985 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-375d8c50-57a3-4da0-8ac2-c16adc57cee0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2384293fd40643bf8e10e0ffefebc9b0', 'request_id': 'req-375d8c50-57a3-4da0-8ac2-c16adc57cee0'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:51.910852 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-375d8c50-57a3-4da0-8ac2-c16adc57cee0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2384293fd40643bf8e10e0ffefebc9b0', 'request_id': 'req-375d8c50-57a3-4da0-8ac2-c16adc57cee0'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:51.929961 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-375d8c50-57a3-4da0-8ac2-c16adc57cee0 admin admin] The token cache is being invalidate because project 2384293fd40643bf8e10e0ffefebc9b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:51.930267 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-375d8c50-57a3-4da0-8ac2-c16adc57cee0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:51.930929 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-375d8c50-57a3-4da0-8ac2-c16adc57cee0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:51.932857 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1406/2833] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:51 2022] DELETE /identity/v3/projects/2384293fd40643bf8e10e0ffefebc9b0 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:51.938050 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a953647-9cd0-46d6-828c-86335887305f None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:51.938345 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a953647-9cd0-46d6-828c-86335887305f None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:51.938672 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a953647-9cd0-46d6-828c-86335887305f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:51.958586 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-7a953647-9cd0-46d6-828c-86335887305f None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:51.959890 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-7a953647-9cd0-46d6-828c-86335887305f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:52.022502 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1428/2834] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:52.030282 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fefe72-6cbe-4216-9b81-118a260a4267 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:52.030666 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fefe72-6cbe-4216-9b81-118a260a4267 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:52.031448 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9fefe72-6cbe-4216-9b81-118a260a4267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:52.051740 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-c9fefe72-6cbe-4216-9b81-118a260a4267 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:52.053022 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-c9fefe72-6cbe-4216-9b81-118a260a4267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:52.061746 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1407/2835] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:52.066878 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a953647-9cd0-46d6-828c-86335887305f None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:52.079063 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a953647-9cd0-46d6-828c-86335887305f None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:52.085428 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038c582-a871-4c04-a7d3-2235e1ab9a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:52.086280 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038c582-a871-4c04-a7d3-2235e1ab9a83 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:52.086612 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038c582-a871-4c04-a7d3-2235e1ab9a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:52.086927 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1038c582-a871-4c04-a7d3-2235e1ab9a83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:52.087713 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038c582-a871-4c04-a7d3-2235e1ab9a83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:08:52.088314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038c582-a871-4c04-a7d3-2235e1ab9a83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:52.090038 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1038c582-a871-4c04-a7d3-2235e1ab9a83 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:52.105975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1429/2836] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:08:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:08:52.110206 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fefe72-6cbe-4216-9b81-118a260a4267 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:52.120026 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9fefe72-6cbe-4216-9b81-118a260a4267 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:52.124875 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a595819-6819-4e74-8826-304c3b838c9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:52.125540 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a595819-6819-4e74-8826-304c3b838c9d admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:52.125827 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a595819-6819-4e74-8826-304c3b838c9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:52.126166 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a595819-6819-4e74-8826-304c3b838c9d admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:52.126896 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a595819-6819-4e74-8826-304c3b838c9d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:52.128818 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a595819-6819-4e74-8826-304c3b838c9d admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:52.149712 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:52.157500 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1408/2837] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:52 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:52.162401 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038c582-a871-4c04-a7d3-2235e1ab9a83 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:52.173010 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1038c582-a871-4c04-a7d3-2235e1ab9a83 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:52.246876 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135bc6d-fda7-458a-8d74-f323ec193c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:52.247837 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135bc6d-fda7-458a-8d74-f323ec193c0c admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:52.248214 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135bc6d-fda7-458a-8d74-f323ec193c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:52.248594 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2135bc6d-fda7-458a-8d74-f323ec193c0c admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:52.249397 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2135bc6d-fda7-458a-8d74-f323ec193c0c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:52.251433 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2135bc6d-fda7-458a-8d74-f323ec193c0c admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:52.275557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1430/2838] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:52 2022] POST /identity/v3/users => generated 292 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:52.276642 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a595819-6819-4e74-8826-304c3b838c9d admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:52.287050 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a595819-6819-4e74-8826-304c3b838c9d admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:52.291992 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ddb17-9505-46a6-914a-c44c49521ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:52.292666 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ddb17-9505-46a6-914a-c44c49521ebe admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:52.292952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ddb17-9505-46a6-914a-c44c49521ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:52.293229 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919ddb17-9505-46a6-914a-c44c49521ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:52.293964 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ddb17-9505-46a6-914a-c44c49521ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:52.295582 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919ddb17-9505-46a6-914a-c44c49521ebe admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:52.307189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1409/2839] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:08:52.311156 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135bc6d-fda7-458a-8d74-f323ec193c0c admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:52.322264 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2135bc6d-fda7-458a-8d74-f323ec193c0c admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:52.327068 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8469c7-9756-46fa-9bf6-13f1276c4b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:52.327730 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8469c7-9756-46fa-9bf6-13f1276c4b79 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:52.328013 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8469c7-9756-46fa-9bf6-13f1276c4b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:52.328288 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8469c7-9756-46fa-9bf6-13f1276c4b79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:52.329038 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8469c7-9756-46fa-9bf6-13f1276c4b79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:52.331216 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8469c7-9756-46fa-9bf6-13f1276c4b79 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:52.344247 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1431/2840] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:52.345401 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ddb17-9505-46a6-914a-c44c49521ebe admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:52.355380 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919ddb17-9505-46a6-914a-c44c49521ebe admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:52.359910 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c5116-b54c-40b4-9e49-2de646232193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:52.360792 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c5116-b54c-40b4-9e49-2de646232193 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:52.361081 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c5116-b54c-40b4-9e49-2de646232193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:52.361359 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79c5116-b54c-40b4-9e49-2de646232193 admin admin] PATH_INFO: `/v3/projects/09a5ddeaa2ee4af483bd6a59a2900cf7/users/4e3d0c2fd023474bb0e4e2df0b14a12c/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:52.378715 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c5116-b54c-40b4-9e49-2de646232193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09a5ddeaa2ee4af483bd6a59a2900cf7, user_id=4e3d0c2fd023474bb0e4e2df0b14a12c, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:52.381375 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79c5116-b54c-40b4-9e49-2de646232193 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:52.396117 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1410/2841] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:52 2022] PUT /identity/v3/projects/09a5ddeaa2ee4af483bd6a59a2900cf7/users/4e3d0c2fd023474bb0e4e2df0b14a12c/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:52.401469 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fad830-c9f5-40df-9928-528226987c05 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:52.401763 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fad830-c9f5-40df-9928-528226987c05 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:52.402044 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fad830-c9f5-40df-9928-528226987c05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:52.420754 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-f8fad830-c9f5-40df-9928-528226987c05 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:52.421934 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-f8fad830-c9f5-40df-9928-528226987c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:52.434660 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:52.482897 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1432/2842] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:52.492506 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c5116-b54c-40b4-9e49-2de646232193 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:52.494223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-b79c5116-b54c-40b4-9e49-2de646232193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:52.551932 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79c5116-b54c-40b4-9e49-2de646232193 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:52.555501 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4482bd7c-1508-491a-9c17-d2eab229a028 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:52.556157 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482bd7c-1508-491a-9c17-d2eab229a028 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:52.556443 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482bd7c-1508-491a-9c17-d2eab229a028 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:52.556722 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4482bd7c-1508-491a-9c17-d2eab229a028 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:52.566057 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482bd7c-1508-491a-9c17-d2eab229a028 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:52.567765 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4482bd7c-1508-491a-9c17-d2eab229a028 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:52.576987 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1411/2843] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:52.962317 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fad830-c9f5-40df-9928-528226987c05 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:52.965190 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-f8fad830-c9f5-40df-9928-528226987c05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:52.984739 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fad830-c9f5-40df-9928-528226987c05 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:53.027158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d81c9e-c880-4d56-8e40-e4fb37bf2208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:53.028067 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d81c9e-c880-4d56-8e40-e4fb37bf2208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:53.028356 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d81c9e-c880-4d56-8e40-e4fb37bf2208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:53.028643 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d81c9e-c880-4d56-8e40-e4fb37bf2208 admin admin] PATH_INFO: `/v3/users/068d4e0774704d86afd61e1d18212101` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:53.031249 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d81c9e-c880-4d56-8e40-e4fb37bf2208 admin admin] RBAC: Authorizing `identity:delete_user(user_id=068d4e0774704d86afd61e1d18212101)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:53.033190 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d81c9e-c880-4d56-8e40-e4fb37bf2208 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:53.139888 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-30d81c9e-c880-4d56-8e40-e4fb37bf2208 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '068d4e0774704d86afd61e1d18212101'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:53.150764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-30d81c9e-c880-4d56-8e40-e4fb37bf2208 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '068d4e0774704d86afd61e1d18212101'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:53.151525 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-30d81c9e-c880-4d56-8e40-e4fb37bf2208 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '068d4e0774704d86afd61e1d18212101'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:53.172042 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-30d81c9e-c880-4d56-8e40-e4fb37bf2208 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '068d4e0774704d86afd61e1d18212101'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:53.173179 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-30d81c9e-c880-4d56-8e40-e4fb37bf2208 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '068d4e0774704d86afd61e1d18212101'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:53.195183 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1433/2844] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:52 2022] DELETE /identity/v3/users/068d4e0774704d86afd61e1d18212101 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:53.196460 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe87fe6-f612-47e5-a8bb-6698ff876383 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:53.196911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe87fe6-f612-47e5-a8bb-6698ff876383 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:53.197037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe87fe6-f612-47e5-a8bb-6698ff876383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:53.217102 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-dbe87fe6-f612-47e5-a8bb-6698ff876383 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:53.218316 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-dbe87fe6-f612-47e5-a8bb-6698ff876383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:53.364858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1412/2845] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:53.384156 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d81c9e-c880-4d56-8e40-e4fb37bf2208 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:53.386677 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-30d81c9e-c880-4d56-8e40-e4fb37bf2208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:53.490701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d81c9e-c880-4d56-8e40-e4fb37bf2208 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:53.495447 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6250ea7-62f0-4843-a3b0-6dcac3502d18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:53.496499 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6250ea7-62f0-4843-a3b0-6dcac3502d18 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:53.496798 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6250ea7-62f0-4843-a3b0-6dcac3502d18 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:53.497084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6250ea7-62f0-4843-a3b0-6dcac3502d18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:53.512414 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6250ea7-62f0-4843-a3b0-6dcac3502d18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:53.514102 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6250ea7-62f0-4843-a3b0-6dcac3502d18 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:53.530739 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1434/2846] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:54.150013 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe87fe6-f612-47e5-a8bb-6698ff876383 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:54.151581 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-dbe87fe6-f612-47e5-a8bb-6698ff876383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:54.174192 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe87fe6-f612-47e5-a8bb-6698ff876383 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:54.197749 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6096e61-71dc-4b1f-95af-e01c9c763d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:54.198681 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6096e61-71dc-4b1f-95af-e01c9c763d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:54.198971 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6096e61-71dc-4b1f-95af-e01c9c763d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:54.199257 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6096e61-71dc-4b1f-95af-e01c9c763d0b admin admin] PATH_INFO: `/v3/projects/478212b6dc6042268691207f6e3d957e` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:54.202880 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6096e61-71dc-4b1f-95af-e01c9c763d0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=478212b6dc6042268691207f6e3d957e)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:54.207458 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6096e61-71dc-4b1f-95af-e01c9c763d0b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:54.310538 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b6096e61-71dc-4b1f-95af-e01c9c763d0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '478212b6dc6042268691207f6e3d957e', 'request_id': 'req-b6096e61-71dc-4b1f-95af-e01c9c763d0b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:54.323432 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b6096e61-71dc-4b1f-95af-e01c9c763d0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '478212b6dc6042268691207f6e3d957e', 'request_id': 'req-b6096e61-71dc-4b1f-95af-e01c9c763d0b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:54.337326 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b6096e61-71dc-4b1f-95af-e01c9c763d0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '478212b6dc6042268691207f6e3d957e', 'request_id': 'req-b6096e61-71dc-4b1f-95af-e01c9c763d0b'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:54.341146 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:54.363382 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b6096e61-71dc-4b1f-95af-e01c9c763d0b admin admin] The token cache is being invalidate because project 478212b6dc6042268691207f6e3d957e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:54.363690 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b6096e61-71dc-4b1f-95af-e01c9c763d0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:54.365451 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b6096e61-71dc-4b1f-95af-e01c9c763d0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:54.367305 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1413/2847] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:54 2022] DELETE /identity/v3/projects/478212b6dc6042268691207f6e3d957e => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:54.372809 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e81a8-5f9e-4bf1-be9e-6530232b2550 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:54.373206 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e81a8-5f9e-4bf1-be9e-6530232b2550 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:54.373319 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04e81a8-5f9e-4bf1-be9e-6530232b2550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:54.412581 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-b04e81a8-5f9e-4bf1-be9e-6530232b2550 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:54.414435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-b04e81a8-5f9e-4bf1-be9e-6530232b2550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:54.503504 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:54.577238 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1435/2848] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:54.586223 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0805516-77d7-477c-bdf1-e7e799464902 None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:54.586528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0805516-77d7-477c-bdf1-e7e799464902 None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:54.586931 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0805516-77d7-477c-bdf1-e7e799464902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:54.619612 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-e0805516-77d7-477c-bdf1-e7e799464902 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:54.621802 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-e0805516-77d7-477c-bdf1-e7e799464902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:54.640148 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e81a8-5f9e-4bf1-be9e-6530232b2550 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:54.641325 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1414/2849] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:54.654108 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04e81a8-5f9e-4bf1-be9e-6530232b2550 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:54.669049 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595fe4b2-7b60-4a23-bce1-274c12325313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:54.672158 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fe4b2-7b60-4a23-bce1-274c12325313 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:54.672362 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fe4b2-7b60-4a23-bce1-274c12325313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:54.672651 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595fe4b2-7b60-4a23-bce1-274c12325313 admin admin] PATH_INFO: `/v3/domains` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:54.673412 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595fe4b2-7b60-4a23-bce1-274c12325313 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80087) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 23:08:54.673733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595fe4b2-7b60-4a23-bce1-274c12325313 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:54.675987 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595fe4b2-7b60-4a23-bce1-274c12325313 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:54.709347 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1436/2850] 10.222.0.17 () {58 vars in 1152 bytes} [Thu Jun 16 23:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 23:08:54.715470 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0805516-77d7-477c-bdf1-e7e799464902 None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:54.729912 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0805516-77d7-477c-bdf1-e7e799464902 None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:54.735357 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71313e28-cff7-4bad-aca6-ac962cde8495 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:54.736207 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71313e28-cff7-4bad-aca6-ac962cde8495 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:54.737172 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71313e28-cff7-4bad-aca6-ac962cde8495 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:54.737702 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71313e28-cff7-4bad-aca6-ac962cde8495 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:54.739078 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71313e28-cff7-4bad-aca6-ac962cde8495 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:54.740798 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71313e28-cff7-4bad-aca6-ac962cde8495 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:54.813759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595fe4b2-7b60-4a23-bce1-274c12325313 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:54.826604 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1415/2851] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:54 2022] POST /identity/v3/projects => generated 371 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:54.832985 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595fe4b2-7b60-4a23-bce1-274c12325313 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:55.004660 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b0882-19c2-493b-9884-d54e8f6c6883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:55.006190 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b0882-19c2-493b-9884-d54e8f6c6883 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:55.006190 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b0882-19c2-493b-9884-d54e8f6c6883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:55.010060 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821b0882-19c2-493b-9884-d54e8f6c6883 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:55.013154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b0882-19c2-493b-9884-d54e8f6c6883 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:55.022568 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821b0882-19c2-493b-9884-d54e8f6c6883 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:55.074232 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1437/2852] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:54 2022] POST /identity/v3/users => generated 286 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:55.078305 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71313e28-cff7-4bad-aca6-ac962cde8495 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:55.089962 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71313e28-cff7-4bad-aca6-ac962cde8495 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:55.104051 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff005761-502d-48ba-9fe6-2d5074e923f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:55.104940 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff005761-502d-48ba-9fe6-2d5074e923f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:55.107218 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff005761-502d-48ba-9fe6-2d5074e923f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:55.107218 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff005761-502d-48ba-9fe6-2d5074e923f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:55.107218 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff005761-502d-48ba-9fe6-2d5074e923f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:55.109553 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff005761-502d-48ba-9fe6-2d5074e923f6 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:55.138699 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1416/2853] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:08:55.140555 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b0882-19c2-493b-9884-d54e8f6c6883 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:55.159454 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821b0882-19c2-493b-9884-d54e8f6c6883 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:55.169495 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5358d21f-57e5-4121-b856-94ec73336649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:55.170193 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358d21f-57e5-4121-b856-94ec73336649 admin admin] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:55.170541 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358d21f-57e5-4121-b856-94ec73336649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:55.171029 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5358d21f-57e5-4121-b856-94ec73336649 admin admin] PATH_INFO: `/v3/roles` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:55.171776 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358d21f-57e5-4121-b856-94ec73336649 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:55.173825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5358d21f-57e5-4121-b856-94ec73336649 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:55.188854 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1438/2854] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:55.190189 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff005761-502d-48ba-9fe6-2d5074e923f6 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:55.203990 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff005761-502d-48ba-9fe6-2d5074e923f6 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:55.218514 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b034b4a-bc1f-4a96-a104-230b78e62567 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:55.219709 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b034b4a-bc1f-4a96-a104-230b78e62567 admin admin] REQUEST_METHOD: `PUT` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:55.220013 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b034b4a-bc1f-4a96-a104-230b78e62567 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:55.220302 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b034b4a-bc1f-4a96-a104-230b78e62567 admin admin] PATH_INFO: `/v3/projects/78d79a495147449aa1a0720d0659342f/users/d3ca8eb2a2bb492eb8961c85086d49e4/roles/bff4f694fb57451e9a0783dc014187f2` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:55.237826 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b034b4a-bc1f-4a96-a104-230b78e62567 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78d79a495147449aa1a0720d0659342f, user_id=d3ca8eb2a2bb492eb8961c85086d49e4, role_id=bff4f694fb57451e9a0783dc014187f2)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:55.240590 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b034b4a-bc1f-4a96-a104-230b78e62567 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:55.255270 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1417/2855] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:55 2022] PUT /identity/v3/projects/78d79a495147449aa1a0720d0659342f/users/d3ca8eb2a2bb492eb8961c85086d49e4/roles/bff4f694fb57451e9a0783dc014187f2 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:55.265399 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e70d2-6c59-4849-b9e3-7745fe400ae7 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:55.265726 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e70d2-6c59-4849-b9e3-7745fe400ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:55.266007 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e70d2-6c59-4849-b9e3-7745fe400ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:55.293514 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-f80e70d2-6c59-4849-b9e3-7745fe400ae7 None None] MFA Rules not processed for user `d3ca8eb2a2bb492eb8961c85086d49e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:55.294788 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-f80e70d2-6c59-4849-b9e3-7745fe400ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:55.363050 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1439/2856] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:55 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:55.376682 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b034b4a-bc1f-4a96-a104-230b78e62567 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:55.380212 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-5b034b4a-bc1f-4a96-a104-230b78e62567 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:55.437794 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b034b4a-bc1f-4a96-a104-230b78e62567 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:55.441938 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec82de3-7c57-4ca6-a625-738f520c49db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:55.442883 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec82de3-7c57-4ca6-a625-738f520c49db service cinder] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:55.443174 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec82de3-7c57-4ca6-a625-738f520c49db service cinder] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:55.443456 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec82de3-7c57-4ca6-a625-738f520c49db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:55.452240 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec82de3-7c57-4ca6-a625-738f520c49db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:55.453673 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec82de3-7c57-4ca6-a625-738f520c49db service cinder] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:55.463469 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1418/2857] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:55.489332 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e70d2-6c59-4849-b9e3-7745fe400ae7 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:55.527007 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:55.567652 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e70d2-6c59-4849-b9e3-7745fe400ae7 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:55.572539 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b9711-2105-4c32-a2fa-1e59be56bc03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:55.573536 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b9711-2105-4c32-a2fa-1e59be56bc03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:55.573898 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b9711-2105-4c32-a2fa-1e59be56bc03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:55.574256 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03b9711-2105-4c32-a2fa-1e59be56bc03 admin admin] PATH_INFO: `/v3/users/d3ca8eb2a2bb492eb8961c85086d49e4` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:55.576624 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b9711-2105-4c32-a2fa-1e59be56bc03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3ca8eb2a2bb492eb8961c85086d49e4)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:55.578439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03b9711-2105-4c32-a2fa-1e59be56bc03 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:55.639429 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-c03b9711-2105-4c32-a2fa-1e59be56bc03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3ca8eb2a2bb492eb8961c85086d49e4'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:55.648146 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-c03b9711-2105-4c32-a2fa-1e59be56bc03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3ca8eb2a2bb492eb8961c85086d49e4'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:55.648885 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-c03b9711-2105-4c32-a2fa-1e59be56bc03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3ca8eb2a2bb492eb8961c85086d49e4'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:55.664220 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-c03b9711-2105-4c32-a2fa-1e59be56bc03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3ca8eb2a2bb492eb8961c85086d49e4'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:55.665090 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-c03b9711-2105-4c32-a2fa-1e59be56bc03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3ca8eb2a2bb492eb8961c85086d49e4'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:55.682100 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1440/2858] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:08:55 2022] DELETE /identity/v3/users/d3ca8eb2a2bb492eb8961c85086d49e4 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:55.686320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f693bda-37d5-4d76-8ac4-f1539795cf2b None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:55.686726 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f693bda-37d5-4d76-8ac4-f1539795cf2b None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:55.687214 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f693bda-37d5-4d76-8ac4-f1539795cf2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:55.736872 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-6f693bda-37d5-4d76-8ac4-f1539795cf2b None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:55.738398 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6f693bda-37d5-4d76-8ac4-f1539795cf2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:55.851152 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1419/2859] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:08:55.873975 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b9711-2105-4c32-a2fa-1e59be56bc03 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:55.875557 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-c03b9711-2105-4c32-a2fa-1e59be56bc03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:55.948899 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03b9711-2105-4c32-a2fa-1e59be56bc03 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:55.952847 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf67f36-17d0-4682-98a3-eb33f41c4ee3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:55.953686 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf67f36-17d0-4682-98a3-eb33f41c4ee3 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:55.954868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf67f36-17d0-4682-98a3-eb33f41c4ee3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:55.955593 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf67f36-17d0-4682-98a3-eb33f41c4ee3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:55.968178 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf67f36-17d0-4682-98a3-eb33f41c4ee3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:55.969983 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf67f36-17d0-4682-98a3-eb33f41c4ee3 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:55.984385 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1441/2860] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:08:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:08:56.693672 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f693bda-37d5-4d76-8ac4-f1539795cf2b None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:56.695763 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-6f693bda-37d5-4d76-8ac4-f1539795cf2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:56.725310 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f693bda-37d5-4d76-8ac4-f1539795cf2b None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:56.732740 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bc813-03e4-4fe6-9242-d90d255a0f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:56.734129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bc813-03e4-4fe6-9242-d90d255a0f35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:56.734129 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bc813-03e4-4fe6-9242-d90d255a0f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:56.734541 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bc813-03e4-4fe6-9242-d90d255a0f35 admin admin] PATH_INFO: `/v3/projects/78d79a495147449aa1a0720d0659342f` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:56.735962 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bc813-03e4-4fe6-9242-d90d255a0f35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78d79a495147449aa1a0720d0659342f)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:56.738519 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bc813-03e4-4fe6-9242-d90d255a0f35 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:56.803195 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:INSERT=3,keystone:DELETE=2,keystone:SELECT=95 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:56.847458 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b14bc813-03e4-4fe6-9242-d90d255a0f35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78d79a495147449aa1a0720d0659342f', 'request_id': 'req-b14bc813-03e4-4fe6-9242-d90d255a0f35'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:56.864365 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b14bc813-03e4-4fe6-9242-d90d255a0f35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78d79a495147449aa1a0720d0659342f', 'request_id': 'req-b14bc813-03e4-4fe6-9242-d90d255a0f35'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:56.873803 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b14bc813-03e4-4fe6-9242-d90d255a0f35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78d79a495147449aa1a0720d0659342f', 'request_id': 'req-b14bc813-03e4-4fe6-9242-d90d255a0f35'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:56.896684 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b14bc813-03e4-4fe6-9242-d90d255a0f35 admin admin] The token cache is being invalidate because project 78d79a495147449aa1a0720d0659342f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:08:56.896979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b14bc813-03e4-4fe6-9242-d90d255a0f35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:56.897650 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-b14bc813-03e4-4fe6-9242-d90d255a0f35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:08:56.899879 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1420/2861] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:08:56 2022] DELETE /identity/v3/projects/78d79a495147449aa1a0720d0659342f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:58.308661 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030610be-bf36-4b2c-a936-7153149c1691 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:58.309136 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030610be-bf36-4b2c-a936-7153149c1691 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:58.309237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030610be-bf36-4b2c-a936-7153149c1691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:58.352524 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-030610be-bf36-4b2c-a936-7153149c1691 None None] MFA Rules not processed for user `4e3d0c2fd023474bb0e4e2df0b14a12c`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:08:58.354951 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-030610be-bf36-4b2c-a936-7153149c1691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:58.427509 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:08:58.469243 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1442/2862] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:08:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:08:58.473329 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bc813-03e4-4fe6-9242-d90d255a0f35 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:58.474868 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-b14bc813-03e4-4fe6-9242-d90d255a0f35 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:08:58.550737 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bc813-03e4-4fe6-9242-d90d255a0f35 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:58.555907 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42031ee-31f5-442f-a13d-250a4d20b641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:58.556683 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42031ee-31f5-442f-a13d-250a4d20b641 admin admin] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:58.557037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42031ee-31f5-442f-a13d-250a4d20b641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:58.557385 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42031ee-31f5-442f-a13d-250a4d20b641 admin admin] PATH_INFO: `/v3/projects` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:58.558164 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42031ee-31f5-442f-a13d-250a4d20b641 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:58.560169 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42031ee-31f5-442f-a13d-250a4d20b641 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:58.588458 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1421/2863] 10.222.0.17 () {60 vars in 1152 bytes} [Thu Jun 16 23:08:58 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:58.592432 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030610be-bf36-4b2c-a936-7153149c1691 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:58.603110 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030610be-bf36-4b2c-a936-7153149c1691 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:58.659237 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de9de6-aeb4-4aba-a74a-3d7c9c214f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:58.659909 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de9de6-aeb4-4aba-a74a-3d7c9c214f23 admin admin] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:58.660213 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de9de6-aeb4-4aba-a74a-3d7c9c214f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:58.660496 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de9de6-aeb4-4aba-a74a-3d7c9c214f23 admin admin] PATH_INFO: `/v3/users` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:58.661176 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80de9de6-aeb4-4aba-a74a-3d7c9c214f23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:58.663138 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80de9de6-aeb4-4aba-a74a-3d7c9c214f23 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:58.683439 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1443/2864] 10.222.0.17 () {60 vars in 1143 bytes} [Thu Jun 16 23:08:58 2022] POST /identity/v3/users => generated 298 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 23:08:58.687486 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42031ee-31f5-442f-a13d-250a4d20b641 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:58.698791 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42031ee-31f5-442f-a13d-250a4d20b641 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:58.703379 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837353ca-adee-4629-8397-2065af801480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:58.704030 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837353ca-adee-4629-8397-2065af801480 admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:58.704320 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837353ca-adee-4629-8397-2065af801480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:58.704603 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837353ca-adee-4629-8397-2065af801480 admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:58.705355 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837353ca-adee-4629-8397-2065af801480 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:58.706982 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837353ca-adee-4629-8397-2065af801480 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:58.717261 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1422/2865] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 23:08:58.721433 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de9de6-aeb4-4aba-a74a-3d7c9c214f23 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:58.731474 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de9de6-aeb4-4aba-a74a-3d7c9c214f23 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:58.736584 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a96b2-6d2b-418c-accf-578c209650a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:58.737477 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a96b2-6d2b-418c-accf-578c209650a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:58.737831 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a96b2-6d2b-418c-accf-578c209650a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:58.738111 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a96b2-6d2b-418c-accf-578c209650a5 admin admin] PATH_INFO: `/v3/projects/e3fa912bc57742ae88c8f464e6843f59/users/3e486fe9a1274ce68282524e5a18ad66/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:58.753356 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a96b2-6d2b-418c-accf-578c209650a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3fa912bc57742ae88c8f464e6843f59, user_id=3e486fe9a1274ce68282524e5a18ad66, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:58.756016 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7a96b2-6d2b-418c-accf-578c209650a5 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:58.770067 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1444/2866] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:58 2022] PUT /identity/v3/projects/e3fa912bc57742ae88c8f464e6843f59/users/3e486fe9a1274ce68282524e5a18ad66/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:08:58.773920 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837353ca-adee-4629-8397-2065af801480 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:58.783790 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837353ca-adee-4629-8397-2065af801480 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:58.839548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6d7e9-6339-4880-8fc3-cdf61be3c67b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:58.840142 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6d7e9-6339-4880-8fc3-cdf61be3c67b admin admin] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:58.840427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6d7e9-6339-4880-8fc3-cdf61be3c67b admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:58.840705 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e6d7e9-6339-4880-8fc3-cdf61be3c67b admin admin] PATH_INFO: `/v3/roles` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:58.841419 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6d7e9-6339-4880-8fc3-cdf61be3c67b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:58.843047 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e6d7e9-6339-4880-8fc3-cdf61be3c67b admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:58.852687 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1423/2867] 10.222.0.17 () {58 vars in 1121 bytes} [Thu Jun 16 23:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 23:08:58.856904 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a96b2-6d2b-418c-accf-578c209650a5 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:08:58.869659 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7a96b2-6d2b-418c-accf-578c209650a5 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:08:58.875306 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b698c74e-ade4-469a-bba4-de7b376aa0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:08:58.876220 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698c74e-ade4-469a-bba4-de7b376aa0a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:08:58.876532 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698c74e-ade4-469a-bba4-de7b376aa0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:08:58.877376 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b698c74e-ade4-469a-bba4-de7b376aa0a8 admin admin] PATH_INFO: `/v3/projects/e3fa912bc57742ae88c8f464e6843f59/users/3e486fe9a1274ce68282524e5a18ad66/roles/bca7a25be0b4426e85d561e14e1bf13e` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:08:58.879444 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698c74e-ade4-469a-bba4-de7b376aa0a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3fa912bc57742ae88c8f464e6843f59, user_id=3e486fe9a1274ce68282524e5a18ad66, role_id=bca7a25be0b4426e85d561e14e1bf13e)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:08:58.882103 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b698c74e-ade4-469a-bba4-de7b376aa0a8 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:08:58.893365 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1445/2868] 10.222.0.17 () {60 vars in 1482 bytes} [Thu Jun 16 23:08:58 2022] PUT /identity/v3/projects/e3fa912bc57742ae88c8f464e6843f59/users/3e486fe9a1274ce68282524e5a18ad66/roles/bca7a25be0b4426e85d561e14e1bf13e => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:09:03.051354 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6d7e9-6339-4880-8fc3-cdf61be3c67b admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:09:03.053180 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-47e6d7e9-6339-4880-8fc3-cdf61be3c67b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:09:03.071869 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e6d7e9-6339-4880-8fc3-cdf61be3c67b admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:09:03.123744 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:09:03.195193 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5c4c3-f934-41a2-98e0-946efdc43aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:09:03.196843 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5c4c3-f934-41a2-98e0-946efdc43aa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:09:03.197280 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5c4c3-f934-41a2-98e0-946efdc43aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:09:03.198247 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba5c4c3-f934-41a2-98e0-946efdc43aa3 admin admin] PATH_INFO: `/v3/users/c50aaf9a551d4f2e8ada9ba762f337aa` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:09:03.203175 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5c4c3-f934-41a2-98e0-946efdc43aa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c50aaf9a551d4f2e8ada9ba762f337aa)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:09:03.206404 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba5c4c3-f934-41a2-98e0-946efdc43aa3 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:09:03.292627 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2ba5c4c3-f934-41a2-98e0-946efdc43aa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c50aaf9a551d4f2e8ada9ba762f337aa'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:09:03.294131 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2ba5c4c3-f934-41a2-98e0-946efdc43aa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c50aaf9a551d4f2e8ada9ba762f337aa'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:09:03.306280 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2ba5c4c3-f934-41a2-98e0-946efdc43aa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c50aaf9a551d4f2e8ada9ba762f337aa'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:09:03.307441 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2ba5c4c3-f934-41a2-98e0-946efdc43aa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c50aaf9a551d4f2e8ada9ba762f337aa'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:09:03.337376 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-2ba5c4c3-f934-41a2-98e0-946efdc43aa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c50aaf9a551d4f2e8ada9ba762f337aa'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:09:03.380440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1424/2869] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:09:03 2022] DELETE /identity/v3/users/c50aaf9a551d4f2e8ada9ba762f337aa => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:09:03.385012 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819c500-dcdc-4388-b7fa-4d28a863b411 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:09:03.385481 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819c500-dcdc-4388-b7fa-4d28a863b411 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:09:03.385712 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0819c500-dcdc-4388-b7fa-4d28a863b411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:09:03.418725 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-0819c500-dcdc-4388-b7fa-4d28a863b411 None None] MFA Rules not processed for user `1288d3a7956c42f7ad18e2e126c7dd07`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:09:03.421473 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-0819c500-dcdc-4388-b7fa-4d28a863b411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:09:03.525544 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1446/2870] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:09:03.538327 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5c4c3-f934-41a2-98e0-946efdc43aa3 admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:09:03.539889 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-2ba5c4c3-f934-41a2-98e0-946efdc43aa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:09:03.562923 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237304b-e509-421b-ad9b-57978b6a254d None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:09:03.563212 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237304b-e509-421b-ad9b-57978b6a254d None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:09:03.563528 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0237304b-e509-421b-ad9b-57978b6a254d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:09:03.624567 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba5c4c3-f934-41a2-98e0-946efdc43aa3 admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:09:03.628855 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef17b8-e351-40bd-a40f-82e780867fe0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:09:03.629828 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef17b8-e351-40bd-a40f-82e780867fe0 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:09:03.630395 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef17b8-e351-40bd-a40f-82e780867fe0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:09:03.630979 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbef17b8-e351-40bd-a40f-82e780867fe0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:09:03.632668 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-0237304b-e509-421b-ad9b-57978b6a254d None None] MFA Rules not processed for user `3e486fe9a1274ce68282524e5a18ad66`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:09:03.634638 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-0237304b-e509-421b-ad9b-57978b6a254d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:09:03.646156 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef17b8-e351-40bd-a40f-82e780867fe0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:09:03.647829 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbef17b8-e351-40bd-a40f-82e780867fe0 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:09:03.660440 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1425/2871] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:09:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:09:03.663958 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:09:03.741930 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1447/2872] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:09:03 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:09:03.761586 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef17b8-e351-40bd-a40f-82e780867fe0 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:09:03.766889 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-bbef17b8-e351-40bd-a40f-82e780867fe0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:09:03.858566 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:09:03.866743 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbef17b8-e351-40bd-a40f-82e780867fe0 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:09:03.875560 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4016151a-7a9a-4fc7-97ed-89aa41b2f3fe service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '915f4296504b47d796e3a5b99550eb75', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:09:03.876504 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016151a-7a9a-4fc7-97ed-89aa41b2f3fe service glance] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:09:03.876905 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016151a-7a9a-4fc7-97ed-89aa41b2f3fe service glance] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:09:03.877476 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4016151a-7a9a-4fc7-97ed-89aa41b2f3fe service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:09:03.880583 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-4016151a-7a9a-4fc7-97ed-89aa41b2f3fe service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:09:03.974724 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016151a-7a9a-4fc7-97ed-89aa41b2f3fe service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:09:03.976445 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4016151a-7a9a-4fc7-97ed-89aa41b2f3fe service glance] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:09:03.996595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1426/2873] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:09:03 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:09:04.144555 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237304b-e509-421b-ad9b-57978b6a254d None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:09:04.146343 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-0237304b-e509-421b-ad9b-57978b6a254d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:09:04.149423 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe96ea8-1da1-4cca-aba2-d2c86033366d None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:09:04.149916 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe96ea8-1da1-4cca-aba2-d2c86033366d None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:09:04.150121 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe96ea8-1da1-4cca-aba2-d2c86033366d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:09:04.172109 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-bbe96ea8-1da1-4cca-aba2-d2c86033366d None None] MFA Rules not processed for user `faf778e713b54a638c1db4a5db6180b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:09:04.173107 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0237304b-e509-421b-ad9b-57978b6a254d None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:09:04.173290 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-bbe96ea8-1da1-4cca-aba2-d2c86033366d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:09:04.179365 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add9d0aa-4181-497e-a49a-8df79afc48bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:09:04.180427 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9d0aa-4181-497e-a49a-8df79afc48bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:09:04.180962 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9d0aa-4181-497e-a49a-8df79afc48bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:09:04.181324 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9d0aa-4181-497e-a49a-8df79afc48bc admin admin] PATH_INFO: `/v3/projects/5fe64a5c67c541419efe1d817ec5c26b` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:09:04.182764 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9d0aa-4181-497e-a49a-8df79afc48bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fe64a5c67c541419efe1d817ec5c26b)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:09:04.184868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add9d0aa-4181-497e-a49a-8df79afc48bc admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:09:04.264388 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-add9d0aa-4181-497e-a49a-8df79afc48bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fe64a5c67c541419efe1d817ec5c26b', 'request_id': 'req-add9d0aa-4181-497e-a49a-8df79afc48bc'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:09:04.269674 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1427/2874] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 23:09:04 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 23:09:04.274321 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-add9d0aa-4181-497e-a49a-8df79afc48bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fe64a5c67c541419efe1d817ec5c26b', 'request_id': 'req-add9d0aa-4181-497e-a49a-8df79afc48bc'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:09:04.286081 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-add9d0aa-4181-497e-a49a-8df79afc48bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fe64a5c67c541419efe1d817ec5c26b', 'request_id': 'req-add9d0aa-4181-497e-a49a-8df79afc48bc'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:09:04.302657 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-add9d0aa-4181-497e-a49a-8df79afc48bc admin admin] The token cache is being invalidate because project 5fe64a5c67c541419efe1d817ec5c26b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80087) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:09:04.303029 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-add9d0aa-4181-497e-a49a-8df79afc48bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:09:04.306759 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-add9d0aa-4181-497e-a49a-8df79afc48bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:09:04.309228 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1448/2875] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:09:04 2022] DELETE /identity/v3/projects/5fe64a5c67c541419efe1d817ec5c26b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:09:04.328926 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe96ea8-1da1-4cca-aba2-d2c86033366d None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:09:04.331757 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-bbe96ea8-1da1-4cca-aba2-d2c86033366d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:09:04.458750 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe96ea8-1da1-4cca-aba2-d2c86033366d None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:09:04.518961 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=100 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:09:04.549228 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcfcd5a-e341-4bee-935b-c71bee521174 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef518c9430e4274b1066241f821f4a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:09:04.549830 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfcd5a-e341-4bee-935b-c71bee521174 service swift] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:09:04.550120 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfcd5a-e341-4bee-935b-c71bee521174 service swift] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:09:04.550402 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcfcd5a-e341-4bee-935b-c71bee521174 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:09:04.552427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-cbcfcd5a-e341-4bee-935b-c71bee521174 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:09:04.624185 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfcd5a-e341-4bee-935b-c71bee521174 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:09:04.625871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbcfcd5a-e341-4bee-935b-c71bee521174 service swift] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:09:04.634659 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1428/2876] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 23:09:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 23:09:13.275128 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a1964-3994-4bbd-89ac-ba88282a81a0 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:09:13.275670 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a1964-3994-4bbd-89ac-ba88282a81a0 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:09:13.275773 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a1964-3994-4bbd-89ac-ba88282a81a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:09:13.293614 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-779a1964-3994-4bbd-89ac-ba88282a81a0 None None] MFA Rules not processed for user `4e3d0c2fd023474bb0e4e2df0b14a12c`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:09:13.295121 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-779a1964-3994-4bbd-89ac-ba88282a81a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:09:13.351088 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1449/2877] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:09:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:09:14.631576 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=32 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:09:15.288281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8601f-e590-4823-88ed-ccbfb9b3933e None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:09:15.288570 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8601f-e590-4823-88ed-ccbfb9b3933e None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:09:15.288848 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd8601f-e590-4823-88ed-ccbfb9b3933e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:09:15.311065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-3dd8601f-e590-4823-88ed-ccbfb9b3933e None None] MFA Rules not processed for user `faf778e713b54a638c1db4a5db6180b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:09:15.312523 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-3dd8601f-e590-4823-88ed-ccbfb9b3933e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:09:15.320799 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1429/2878] 10.222.0.17 () {58 vars in 1009 bytes} [Thu Jun 16 23:09:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:09:15.336598 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a1964-3994-4bbd-89ac-ba88282a81a0 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:09:15.339753 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-779a1964-3994-4bbd-89ac-ba88282a81a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:09:15.356472 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a1964-3994-4bbd-89ac-ba88282a81a0 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:09:15.362082 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fce29-1470-4c25-a90b-611c49aac880 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef518c9430e4274b1066241f821f4a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:09:15.362782 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fce29-1470-4c25-a90b-611c49aac880 service swift] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:09:15.363073 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fce29-1470-4c25-a90b-611c49aac880 service swift] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:09:15.363400 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185fce29-1470-4c25-a90b-611c49aac880 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:09:15.372599 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fce29-1470-4c25-a90b-611c49aac880 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:09:15.374181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185fce29-1470-4c25-a90b-611c49aac880 service swift] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:09:15.382787 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1450/2879] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 23:09:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 23:09:25.305947 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=3 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:09:25.379209 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:09:39.581815 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd8601f-e590-4823-88ed-ccbfb9b3933e None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:09:39.583345 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-3dd8601f-e590-4823-88ed-ccbfb9b3933e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:09:39.638826 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd8601f-e590-4823-88ed-ccbfb9b3933e None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:09:39.642521 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9407e-ee60-4c2e-9b79-515d8b9ce995 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:09:39.643196 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9407e-ee60-4c2e-9b79-515d8b9ce995 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:09:39.643508 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9407e-ee60-4c2e-9b79-515d8b9ce995 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:09:39.643788 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9407e-ee60-4c2e-9b79-515d8b9ce995 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:09:39.645530 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-8ba9407e-ee60-4c2e-9b79-515d8b9ce995 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:09:39.706159 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9407e-ee60-4c2e-9b79-515d8b9ce995 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:09:39.707612 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9407e-ee60-4c2e-9b79-515d8b9ce995 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:09:39.717341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1430/2880] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:09:39 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:09:49.712671 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=50 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:10:52.126701 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fce29-1470-4c25-a90b-611c49aac880 service swift] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:10:52.138921 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185fce29-1470-4c25-a90b-611c49aac880 service swift] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:10:52.142912 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29948de-14e8-4645-a895-a6351778d2c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:10:52.143555 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29948de-14e8-4645-a895-a6351778d2c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:10:52.143842 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29948de-14e8-4645-a895-a6351778d2c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:10:52.144122 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29948de-14e8-4645-a895-a6351778d2c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:10:52.146017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a29948de-14e8-4645-a895-a6351778d2c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:10:52.219653 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29948de-14e8-4645-a895-a6351778d2c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:10:52.221080 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29948de-14e8-4645-a895-a6351778d2c8 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:10:52.232868 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1451/2881] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:10:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:10:54.859694 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9407e-ee60-4c2e-9b79-515d8b9ce995 service neutron] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:10:54.861107 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-8ba9407e-ee60-4c2e-9b79-515d8b9ce995 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:10:54.872178 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9407e-ee60-4c2e-9b79-515d8b9ce995 service neutron] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:10:54.875707 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d26e7-01b4-452e-a60b-8c68b2961399 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:10:54.876352 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d26e7-01b4-452e-a60b-8c68b2961399 service neutron] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:10:54.876634 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d26e7-01b4-452e-a60b-8c68b2961399 service neutron] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:10:54.876911 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d26e7-01b4-452e-a60b-8c68b2961399 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:10:54.878650 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-413d26e7-01b4-452e-a60b-8c68b2961399 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:10:54.887819 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d26e7-01b4-452e-a60b-8c68b2961399 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:10:54.889224 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413d26e7-01b4-452e-a60b-8c68b2961399 service neutron] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:10:54.897056 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1431/2882] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:10:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 23:10:56.214788 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29948de-14e8-4645-a895-a6351778d2c8 service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:10:56.216211 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a29948de-14e8-4645-a895-a6351778d2c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:10:56.272184 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29948de-14e8-4645-a895-a6351778d2c8 service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:10:56.275724 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6faba-95b0-4380-bb3c-12cb25e2fe44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24ca7f221334442781f10574277bf422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:10:56.276371 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6faba-95b0-4380-bb3c-12cb25e2fe44 service cinder] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:10:56.276659 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6faba-95b0-4380-bb3c-12cb25e2fe44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:10:56.276941 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb6faba-95b0-4380-bb3c-12cb25e2fe44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:10:56.278710 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-9cb6faba-95b0-4380-bb3c-12cb25e2fe44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:10:56.292379 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6faba-95b0-4380-bb3c-12cb25e2fe44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:10:56.293806 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb6faba-95b0-4380-bb3c-12cb25e2fe44 service cinder] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:10:56.305251 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1452/2883] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:10:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:10:56.770766 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aec2f4-4ba1-4b69-abf8-01fd4efbf8bc None None] REQUEST_METHOD: `POST` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:10:56.771282 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aec2f4-4ba1-4b69-abf8-01fd4efbf8bc None None] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:10:56.771392 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aec2f4-4ba1-4b69-abf8-01fd4efbf8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:10:56.788459 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.auth.core [None req-29aec2f4-4ba1-4b69-abf8-01fd4efbf8bc None None] MFA Rules not processed for user `2697aa77265e4f5783f87ffb3d4992b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80088) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:10:56.789653 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-29aec2f4-4ba1-4b69-abf8-01fd4efbf8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:10:56.844287 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1432/2884] 10.222.0.17 () {58 vars in 957 bytes} [Thu Jun 16 23:10:56 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:10:57.575967 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6faba-95b0-4380-bb3c-12cb25e2fe44 service cinder] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:10:57.577376 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-9cb6faba-95b0-4380-bb3c-12cb25e2fe44 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:10:57.669456 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb6faba-95b0-4380-bb3c-12cb25e2fe44 service cinder] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:10:57.674856 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b80a035-f7f7-414f-a8c8-9a754d73574a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9593939641c4c7797c1981206b40e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:10:57.675510 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80a035-f7f7-414f-a8c8-9a754d73574a service placement] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:10:57.675802 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80a035-f7f7-414f-a8c8-9a754d73574a service placement] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:10:57.676085 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b80a035-f7f7-414f-a8c8-9a754d73574a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:10:57.677899 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-0b80a035-f7f7-414f-a8c8-9a754d73574a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:10:57.690117 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80a035-f7f7-414f-a8c8-9a754d73574a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:10:57.691734 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b80a035-f7f7-414f-a8c8-9a754d73574a service placement] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:10:57.702519 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1453/2885] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:10:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:10:58.649538 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aec2f4-4ba1-4b69-abf8-01fd4efbf8bc None None] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:10:58.651065 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-29aec2f4-4ba1-4b69-abf8-01fd4efbf8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:10:58.667021 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aec2f4-4ba1-4b69-abf8-01fd4efbf8bc None None] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:10:58.672337 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463a6fa-f3e3-49ee-9387-e7a1f9343ed2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:10:58.672969 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463a6fa-f3e3-49ee-9387-e7a1f9343ed2 service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:10:58.673253 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463a6fa-f3e3-49ee-9387-e7a1f9343ed2 service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:10:58.673528 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0463a6fa-f3e3-49ee-9387-e7a1f9343ed2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:10:58.675225 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-0463a6fa-f3e3-49ee-9387-e7a1f9343ed2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:10:58.685936 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463a6fa-f3e3-49ee-9387-e7a1f9343ed2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:10:58.687520 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0463a6fa-f3e3-49ee-9387-e7a1f9343ed2 service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:10:58.696888 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1433/2886] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:10:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:11:01.220554 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06df5f-16f1-4ac1-8e44-c7cd36567378 None None] REQUEST_METHOD: `POST` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:11:01.221084 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06df5f-16f1-4ac1-8e44-c7cd36567378 None None] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:11:01.221201 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c06df5f-16f1-4ac1-8e44-c7cd36567378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:11:01.243546 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.auth.core [None req-8c06df5f-16f1-4ac1-8e44-c7cd36567378 None None] MFA Rules not processed for user `faf778e713b54a638c1db4a5db6180b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80087) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 23:11:01.244743 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-8c06df5f-16f1-4ac1-8e44-c7cd36567378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:11:01.252678 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1454/2887] 10.222.0.17 () {58 vars in 967 bytes} [Thu Jun 16 23:11:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 23:11:01.298113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463a6fa-f3e3-49ee-9387-e7a1f9343ed2 service nova] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:11:01.310477 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0463a6fa-f3e3-49ee-9387-e7a1f9343ed2 service nova] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:11:01.316248 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee586a-3a79-498b-ab14-0f3047175cbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bef518c9430e4274b1066241f821f4a1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:11:01.317315 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee586a-3a79-498b-ab14-0f3047175cbf service swift] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:11:01.317730 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee586a-3a79-498b-ab14-0f3047175cbf service swift] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:11:01.318107 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee586a-3a79-498b-ab14-0f3047175cbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:11:01.329467 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee586a-3a79-498b-ab14-0f3047175cbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:11:01.331308 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee586a-3a79-498b-ab14-0f3047175cbf service swift] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:11:01.341754 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1434/2888] 10.222.0.17 () {60 vars in 1384 bytes} [Thu Jun 16 23:11:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 23:11:08.587001 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06df5f-16f1-4ac1-8e44-c7cd36567378 None None] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:11:08.588581 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-8c06df5f-16f1-4ac1-8e44-c7cd36567378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:11:08.604333 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=100 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:11:08.659879 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c06df5f-16f1-4ac1-8e44-c7cd36567378 None None] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:11:08.666352 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd22058-dccb-4b42-a638-ad3096704df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:11:08.667282 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd22058-dccb-4b42-a638-ad3096704df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:11:08.667571 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd22058-dccb-4b42-a638-ad3096704df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:11:08.667853 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd22058-dccb-4b42-a638-ad3096704df3 admin admin] PATH_INFO: `/v3/users/4e3d0c2fd023474bb0e4e2df0b14a12c` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:11:08.669154 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd22058-dccb-4b42-a638-ad3096704df3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e3d0c2fd023474bb0e4e2df0b14a12c)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:11:08.671174 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd22058-dccb-4b42-a638-ad3096704df3 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:11:08.722610 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-cbd22058-dccb-4b42-a638-ad3096704df3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e3d0c2fd023474bb0e4e2df0b14a12c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:08.729305 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-cbd22058-dccb-4b42-a638-ad3096704df3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e3d0c2fd023474bb0e4e2df0b14a12c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:08.729967 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-cbd22058-dccb-4b42-a638-ad3096704df3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e3d0c2fd023474bb0e4e2df0b14a12c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:08.741747 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-cbd22058-dccb-4b42-a638-ad3096704df3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e3d0c2fd023474bb0e4e2df0b14a12c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:08.742406 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-cbd22058-dccb-4b42-a638-ad3096704df3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e3d0c2fd023474bb0e4e2df0b14a12c'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:08.755585 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1455/2889] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:11:08 2022] DELETE /identity/v3/users/4e3d0c2fd023474bb0e4e2df0b14a12c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:11:09.019624 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee586a-3a79-498b-ab14-0f3047175cbf service swift] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:11:09.033281 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee586a-3a79-498b-ab14-0f3047175cbf service swift] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:11:09.093312 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54835d-1590-4d34-ad5d-a8475839084e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:11:09.094177 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54835d-1590-4d34-ad5d-a8475839084e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:11:09.094463 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54835d-1590-4d34-ad5d-a8475839084e admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:11:09.094774 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d54835d-1590-4d34-ad5d-a8475839084e admin admin] PATH_INFO: `/v3/projects/09a5ddeaa2ee4af483bd6a59a2900cf7` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:11:09.095952 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54835d-1590-4d34-ad5d-a8475839084e admin admin] RBAC: Authorizing `identity:delete_project(project_id=09a5ddeaa2ee4af483bd6a59a2900cf7)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:11:09.097839 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d54835d-1590-4d34-ad5d-a8475839084e admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:11:09.145544 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1d54835d-1590-4d34-ad5d-a8475839084e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09a5ddeaa2ee4af483bd6a59a2900cf7', 'request_id': 'req-1d54835d-1590-4d34-ad5d-a8475839084e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:09.151537 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1d54835d-1590-4d34-ad5d-a8475839084e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09a5ddeaa2ee4af483bd6a59a2900cf7', 'request_id': 'req-1d54835d-1590-4d34-ad5d-a8475839084e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:09.157595 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1d54835d-1590-4d34-ad5d-a8475839084e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09a5ddeaa2ee4af483bd6a59a2900cf7', 'request_id': 'req-1d54835d-1590-4d34-ad5d-a8475839084e'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:09.161180 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:11:09.171064 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1d54835d-1590-4d34-ad5d-a8475839084e admin admin] The token cache is being invalidate because project 09a5ddeaa2ee4af483bd6a59a2900cf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:11:09.171513 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1d54835d-1590-4d34-ad5d-a8475839084e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:09.172425 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-1d54835d-1590-4d34-ad5d-a8475839084e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:09.174011 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1435/2890] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:11:09 2022] DELETE /identity/v3/projects/09a5ddeaa2ee4af483bd6a59a2900cf7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:11:09.178562 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd22058-dccb-4b42-a638-ad3096704df3 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:11:09.180979 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-cbd22058-dccb-4b42-a638-ad3096704df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:11:09.193748 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd22058-dccb-4b42-a638-ad3096704df3 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:11:09.258535 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b98e4-f095-40dd-a764-82dea986e8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:11:09.259506 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b98e4-f095-40dd-a764-82dea986e8e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:11:09.259799 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b98e4-f095-40dd-a764-82dea986e8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:11:09.260079 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993b98e4-f095-40dd-a764-82dea986e8e3 admin admin] PATH_INFO: `/v3/users/3e486fe9a1274ce68282524e5a18ad66` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:11:09.261328 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b98e4-f095-40dd-a764-82dea986e8e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e486fe9a1274ce68282524e5a18ad66)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:11:09.263367 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993b98e4-f095-40dd-a764-82dea986e8e3 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:11:09.300301 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:11:09.316160 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-993b98e4-f095-40dd-a764-82dea986e8e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e486fe9a1274ce68282524e5a18ad66'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:09.324823 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-993b98e4-f095-40dd-a764-82dea986e8e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e486fe9a1274ce68282524e5a18ad66'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:09.325598 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-993b98e4-f095-40dd-a764-82dea986e8e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e486fe9a1274ce68282524e5a18ad66'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:09.338150 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-993b98e4-f095-40dd-a764-82dea986e8e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e486fe9a1274ce68282524e5a18ad66'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:09.338841 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-993b98e4-f095-40dd-a764-82dea986e8e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e486fe9a1274ce68282524e5a18ad66'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:09.351761 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1456/2891] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:11:09 2022] DELETE /identity/v3/users/3e486fe9a1274ce68282524e5a18ad66 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:11:09.523421 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54835d-1590-4d34-ad5d-a8475839084e admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:11:09.525113 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-1d54835d-1590-4d34-ad5d-a8475839084e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:11:09.539813 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d54835d-1590-4d34-ad5d-a8475839084e admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:11:09.598112 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f403a-18d5-489f-9a28-563882ea853a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:11:09.599014 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f403a-18d5-489f-9a28-563882ea853a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:11:09.599307 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f403a-18d5-489f-9a28-563882ea853a admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:11:09.599589 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9f403a-18d5-489f-9a28-563882ea853a admin admin] PATH_INFO: `/v3/projects/e3fa912bc57742ae88c8f464e6843f59` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:11:09.600799 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f403a-18d5-489f-9a28-563882ea853a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3fa912bc57742ae88c8f464e6843f59)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:11:09.602750 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9f403a-18d5-489f-9a28-563882ea853a admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:11:09.648048 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-df9f403a-18d5-489f-9a28-563882ea853a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3fa912bc57742ae88c8f464e6843f59', 'request_id': 'req-df9f403a-18d5-489f-9a28-563882ea853a'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:09.654157 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-df9f403a-18d5-489f-9a28-563882ea853a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3fa912bc57742ae88c8f464e6843f59', 'request_id': 'req-df9f403a-18d5-489f-9a28-563882ea853a'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:09.659720 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-df9f403a-18d5-489f-9a28-563882ea853a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3fa912bc57742ae88c8f464e6843f59', 'request_id': 'req-df9f403a-18d5-489f-9a28-563882ea853a'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:09.671565 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-df9f403a-18d5-489f-9a28-563882ea853a admin admin] The token cache is being invalidate because project e3fa912bc57742ae88c8f464e6843f59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:11:09.671858 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-df9f403a-18d5-489f-9a28-563882ea853a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:09.672491 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-df9f403a-18d5-489f-9a28-563882ea853a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:11:09.674005 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1436/2892] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:11:09 2022] DELETE /identity/v3/projects/e3fa912bc57742ae88c8f464e6843f59 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:11:14.485530 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b98e4-f095-40dd-a764-82dea986e8e3 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:11:14.486959 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-993b98e4-f095-40dd-a764-82dea986e8e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:11:14.562550 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993b98e4-f095-40dd-a764-82dea986e8e3 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:11:14.626119 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449e3e1-fa8e-49d6-9fc6-cabde02691dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:11:14.626825 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449e3e1-fa8e-49d6-9fc6-cabde02691dd service nova] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:11:14.627119 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449e3e1-fa8e-49d6-9fc6-cabde02691dd service nova] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:11:14.627405 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1449e3e1-fa8e-49d6-9fc6-cabde02691dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:11:14.629192 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-1449e3e1-fa8e-49d6-9fc6-cabde02691dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:11:14.641883 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1449e3e1-fa8e-49d6-9fc6-cabde02691dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:11:14.643507 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1449e3e1-fa8e-49d6-9fc6-cabde02691dd service nova] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:11:14.654389 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1457/2893] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:11:14 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:11:19.668621 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:11:24.650034 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=79,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:12:45.526037 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f403a-18d5-489f-9a28-563882ea853a admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:12:45.527548 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-df9f403a-18d5-489f-9a28-563882ea853a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:12:45.544421 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9f403a-18d5-489f-9a28-563882ea853a admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:12:45.550452 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d3582-eae3-43e6-bec8-a178a041f248 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4180f2593149437fa27effd8d1338d10', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'service', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:12:45.551139 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d3582-eae3-43e6-bec8-a178a041f248 service nova] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:12:45.551427 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d3582-eae3-43e6-bec8-a178a041f248 service nova] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:12:45.552210 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713d3582-eae3-43e6-bec8-a178a041f248 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:12:45.554279 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-713d3582-eae3-43e6-bec8-a178a041f248 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:12:45.566965 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d3582-eae3-43e6-bec8-a178a041f248 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:12:45.569207 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713d3582-eae3-43e6-bec8-a178a041f248 service nova] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:12:45.580443 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1437/2894] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:12:45 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:12:55.525852 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449e3e1-fa8e-49d6-9fc6-cabde02691dd service nova] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:12:55.527314 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-1449e3e1-fa8e-49d6-9fc6-cabde02691dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:12:55.575635 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=6 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:12:55.582479 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1449e3e1-fa8e-49d6-9fc6-cabde02691dd service nova] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:12:55.586056 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db7687-ed96-40e8-a949-7e491a9640d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6b794f7aa2a7488e91f7d9ea35a45d78', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:12:55.586731 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db7687-ed96-40e8-a949-7e491a9640d9 service neutron] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:12:55.587017 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db7687-ed96-40e8-a949-7e491a9640d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:12:55.587293 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7db7687-ed96-40e8-a949-7e491a9640d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:12:55.589024 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a7db7687-ed96-40e8-a949-7e491a9640d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:12:55.649579 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db7687-ed96-40e8-a949-7e491a9640d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:12:55.651190 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7db7687-ed96-40e8-a949-7e491a9640d9 service neutron] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:12:55.661272 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1458/2895] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:12:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:12:58.631960 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d3582-eae3-43e6-bec8-a178a041f248 service nova] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:12:58.633613 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-713d3582-eae3-43e6-bec8-a178a041f248 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:12:58.724607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713d3582-eae3-43e6-bec8-a178a041f248 service nova] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:12:58.730886 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f5ca5-8db9-4f3a-b5a2-9ad9e30cc190 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9593939641c4c7797c1981206b40e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:12:58.731561 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f5ca5-8db9-4f3a-b5a2-9ad9e30cc190 service placement] REQUEST_METHOD: `GET` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:12:58.731847 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f5ca5-8db9-4f3a-b5a2-9ad9e30cc190 service placement] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:12:58.732126 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f5ca5-8db9-4f3a-b5a2-9ad9e30cc190 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:12:58.733970 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-7b2f5ca5-8db9-4f3a-b5a2-9ad9e30cc190 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:12:58.746388 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f5ca5-8db9-4f3a-b5a2-9ad9e30cc190 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:12:58.748109 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f5ca5-8db9-4f3a-b5a2-9ad9e30cc190 service placement] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:12:58.759108 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1438/2896] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:12:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:13:01.497280 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db7687-ed96-40e8-a949-7e491a9640d9 service neutron] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:13:01.500668 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-a7db7687-ed96-40e8-a949-7e491a9640d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:13:01.518517 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7db7687-ed96-40e8-a949-7e491a9640d9 service neutron] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:13:01.524367 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c06b45-62e9-4d0f-9e16-92a74ee12289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:13:01.525353 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c06b45-62e9-4d0f-9e16-92a74ee12289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:13:01.525707 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c06b45-62e9-4d0f-9e16-92a74ee12289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:13:01.525989 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c06b45-62e9-4d0f-9e16-92a74ee12289 admin admin] PATH_INFO: `/v3/users/2697aa77265e4f5783f87ffb3d4992b5` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:13:01.527377 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c06b45-62e9-4d0f-9e16-92a74ee12289 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2697aa77265e4f5783f87ffb3d4992b5)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:13:01.529322 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c06b45-62e9-4d0f-9e16-92a74ee12289 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:13:01.582667 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-66c06b45-62e9-4d0f-9e16-92a74ee12289 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2697aa77265e4f5783f87ffb3d4992b5'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:01.589465 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-66c06b45-62e9-4d0f-9e16-92a74ee12289 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2697aa77265e4f5783f87ffb3d4992b5'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:01.590130 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-66c06b45-62e9-4d0f-9e16-92a74ee12289 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2697aa77265e4f5783f87ffb3d4992b5'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:01.601565 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-66c06b45-62e9-4d0f-9e16-92a74ee12289 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2697aa77265e4f5783f87ffb3d4992b5'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:01.602181 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-66c06b45-62e9-4d0f-9e16-92a74ee12289 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2697aa77265e4f5783f87ffb3d4992b5'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:01.614972 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1459/2897] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:13:01 2022] DELETE /identity/v3/users/2697aa77265e4f5783f87ffb3d4992b5 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:13:01.784145 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f5ca5-8db9-4f3a-b5a2-9ad9e30cc190 service placement] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:13:01.800409 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f5ca5-8db9-4f3a-b5a2-9ad9e30cc190 service placement] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:13:01.860432 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946366b-8ed5-49fa-ae89-914af93a2b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:13:01.861300 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946366b-8ed5-49fa-ae89-914af93a2b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:13:01.861590 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946366b-8ed5-49fa-ae89-914af93a2b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:13:01.861871 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d946366b-8ed5-49fa-ae89-914af93a2b2d admin admin] PATH_INFO: `/v3/projects/1423c05c42b14519b401e22e16062781` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:13:01.863617 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946366b-8ed5-49fa-ae89-914af93a2b2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=1423c05c42b14519b401e22e16062781)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:13:01.865513 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d946366b-8ed5-49fa-ae89-914af93a2b2d admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:13:01.911593 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d946366b-8ed5-49fa-ae89-914af93a2b2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1423c05c42b14519b401e22e16062781', 'request_id': 'req-d946366b-8ed5-49fa-ae89-914af93a2b2d'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:01.917649 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d946366b-8ed5-49fa-ae89-914af93a2b2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1423c05c42b14519b401e22e16062781', 'request_id': 'req-d946366b-8ed5-49fa-ae89-914af93a2b2d'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:01.923625 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d946366b-8ed5-49fa-ae89-914af93a2b2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1423c05c42b14519b401e22e16062781', 'request_id': 'req-d946366b-8ed5-49fa-ae89-914af93a2b2d'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:01.938297 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d946366b-8ed5-49fa-ae89-914af93a2b2d admin admin] The token cache is being invalidate because project 1423c05c42b14519b401e22e16062781 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:13:01.938612 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d946366b-8ed5-49fa-ae89-914af93a2b2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:01.939269 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-d946366b-8ed5-49fa-ae89-914af93a2b2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:01.941482 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1439/2898] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:13:01 2022] DELETE /identity/v3/projects/1423c05c42b14519b401e22e16062781 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:13:01.945413 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c06b45-62e9-4d0f-9e16-92a74ee12289 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:13:01.946905 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-66c06b45-62e9-4d0f-9e16-92a74ee12289 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:13:01.961159 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c06b45-62e9-4d0f-9e16-92a74ee12289 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:13:01.980588 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:13:02.021435 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5234e0-cec0-46cc-9b15-ad9afa7fdb81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:13:02.022410 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5234e0-cec0-46cc-9b15-ad9afa7fdb81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:13:02.022733 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5234e0-cec0-46cc-9b15-ad9afa7fdb81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:13:02.023012 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5234e0-cec0-46cc-9b15-ad9afa7fdb81 admin admin] PATH_INFO: `/v3/users/0dadd97b13ce42d8b0954a401cdd1b67` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:13:02.024211 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5234e0-cec0-46cc-9b15-ad9afa7fdb81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0dadd97b13ce42d8b0954a401cdd1b67)` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:13:02.027968 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5234e0-cec0-46cc-9b15-ad9afa7fdb81 admin admin] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:13:02.079587 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-ae5234e0-cec0-46cc-9b15-ad9afa7fdb81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0dadd97b13ce42d8b0954a401cdd1b67'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:02.087083 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-ae5234e0-cec0-46cc-9b15-ad9afa7fdb81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0dadd97b13ce42d8b0954a401cdd1b67'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:02.087777 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-ae5234e0-cec0-46cc-9b15-ad9afa7fdb81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0dadd97b13ce42d8b0954a401cdd1b67'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:02.099452 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-ae5234e0-cec0-46cc-9b15-ad9afa7fdb81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0dadd97b13ce42d8b0954a401cdd1b67'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:02.100079 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.notifications [None req-ae5234e0-cec0-46cc-9b15-ad9afa7fdb81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0dadd97b13ce42d8b0954a401cdd1b67'} {{(pid=80087) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:02.112391 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1460/2899] 10.222.0.17 () {58 vars in 1223 bytes} [Thu Jun 16 23:13:01 2022] DELETE /identity/v3/users/0dadd97b13ce42d8b0954a401cdd1b67 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 23:13:02.281849 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946366b-8ed5-49fa-ae89-914af93a2b2d admin admin] Authenticating user token {{(pid=80088) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:13:02.283761 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.fernet_utils [None req-d946366b-8ed5-49fa-ae89-914af93a2b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80088) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:13:02.296853 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d946366b-8ed5-49fa-ae89-914af93a2b2d admin admin] Validating token access rules against request {{(pid=80088) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:13:02.314179 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:13:02.355741 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc26ece8-d01b-45a4-9e04-8df0d72710b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1288d3a7956c42f7ad18e2e126c7dd07', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2ffbb7bcd828426c857e7fce279519f7', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80088) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:13:02.356607 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26ece8-d01b-45a4-9e04-8df0d72710b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:13:02.356893 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26ece8-d01b-45a4-9e04-8df0d72710b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:13:02.357171 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc26ece8-d01b-45a4-9e04-8df0d72710b2 admin admin] PATH_INFO: `/v3/projects/3611b63c998b4abebe0a7deb01d1f0d5` {{(pid=80088) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:13:02.358341 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26ece8-d01b-45a4-9e04-8df0d72710b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3611b63c998b4abebe0a7deb01d1f0d5)` {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:13:02.360291 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc26ece8-d01b-45a4-9e04-8df0d72710b2 admin admin] RBAC: Authorization granted {{(pid=80088) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:13:02.405680 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-bc26ece8-d01b-45a4-9e04-8df0d72710b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3611b63c998b4abebe0a7deb01d1f0d5', 'request_id': 'req-bc26ece8-d01b-45a4-9e04-8df0d72710b2'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:02.411668 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-bc26ece8-d01b-45a4-9e04-8df0d72710b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3611b63c998b4abebe0a7deb01d1f0d5', 'request_id': 'req-bc26ece8-d01b-45a4-9e04-8df0d72710b2'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:02.417117 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-bc26ece8-d01b-45a4-9e04-8df0d72710b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3611b63c998b4abebe0a7deb01d1f0d5', 'request_id': 'req-bc26ece8-d01b-45a4-9e04-8df0d72710b2'} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:02.429300 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-bc26ece8-d01b-45a4-9e04-8df0d72710b2 admin admin] The token cache is being invalidate because project 3611b63c998b4abebe0a7deb01d1f0d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80088) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 23:13:02.429589 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-bc26ece8-d01b-45a4-9e04-8df0d72710b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:02.430216 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG keystone.notifications [None req-bc26ece8-d01b-45a4-9e04-8df0d72710b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80088) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 23:13:02.431770 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: [pid: 80088|app: 0|req: 1440/2900] 10.222.0.17 () {58 vars in 1232 bytes} [Thu Jun 16 23:13:02 2022] DELETE /identity/v3/projects/3611b63c998b4abebe0a7deb01d1f0d5 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 23:13:12.108742 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=43,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:13:12.426264 dv-cinder-846230-1-iscsi devstack@keystone.service[80088]: DEBUG dbcounter [-] [80088] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80088) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 23:13:50.802537 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5234e0-cec0-46cc-9b15-ad9afa7fdb81 admin admin] Authenticating user token {{(pid=80087) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 23:13:50.803966 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-ae5234e0-cec0-46cc-9b15-ad9afa7fdb81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:13:50.942599 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5234e0-cec0-46cc-9b15-ad9afa7fdb81 admin admin] Validating token access rules against request {{(pid=80087) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 23:13:51.000299 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f249e7-9480-4d99-922a-c0a86d1253fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f9593939641c4c7797c1981206b40e7d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bc48914270df4db996bd2a4b5b9fc777', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80087) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 23:13:51.001447 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f249e7-9480-4d99-922a-c0a86d1253fe service placement] REQUEST_METHOD: `GET` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 23:13:51.001732 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f249e7-9480-4d99-922a-c0a86d1253fe service placement] SCRIPT_NAME: `/identity` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 23:13:51.002012 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f249e7-9480-4d99-922a-c0a86d1253fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80087) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 23:13:51.003789 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.fernet_utils [None req-51f249e7-9480-4d99-922a-c0a86d1253fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80087) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 23:13:51.014626 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f249e7-9480-4d99-922a-c0a86d1253fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 23:13:51.016176 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f249e7-9480-4d99-922a-c0a86d1253fe service placement] RBAC: Authorization granted {{(pid=80087) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 23:13:51.026457 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: [pid: 80087|app: 0|req: 1461/2901] 10.222.0.17 () {60 vars in 1365 bytes} [Thu Jun 16 23:13:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 23:14:01.022217 dv-cinder-846230-1-iscsi devstack@keystone.service[80087]: DEBUG dbcounter [-] [80087] Writing DB stats keystone:SELECT=89 {{(pid=80087) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}